[squid-users] Rock store limit

[Alex Rousskov]

On 2024-04-16 03:20, FredB wrote:

> I'm trying to use rock store with 6.9, there is a limitation about the 
> size of cache ?

If my calculations are correct, all cache_dirs share the same byte-size 
limit: A single cache_dir cannot store more than ~2048 terabytes (i.e. 
2^51 bytes).

However, all cache_dir types are also limited by factors other than the 
total size. For example, each cache_dir cannot store more than 
16'777'215 entries (2^24-1).

IIRC, rock cache_dirs also cannot have more than 2'147'483'648 slots 
(2^31). See cache_dir rock ... slot-size=bytes documentation for more 
info regarding rock slots.

Rock cache_dirs are also limited by the maximum shared memory segment 
size. A cache_dir index maintains an index in shared memory. That index 
has three components. Each component must fit into a dedicated shared 
memory segment (your OS configuration limits the size of that segment):

* index component A:  4 bytes per cache_dir entry
* index component B: 96 bytes per cache_dir entry
* index component C:  8 bytes per cache_dir slot


> I tried 15000 but there is no rock db created with squid -z

What error do you get from Squid? Please note that Squid may be limited 
by the maximum shared memory segment size or some other limit.


> My goal is using a 200G SSD disk

With default store_avg_object_size of 13KB, the 16'777'215 entry limit 
implies the maximum cache_dir size of ~208G, but, again, there are other 
limits.


Please also note that large rock cache_dirs will take a long time to be 
indexed on Squid startup. Rock indexing is usually done in background, 
but it is still a significant performance expense. Optimizing indexing 
is an old item on our to-do list.


HTH,

Alex.


> cache_dir rock /cache 1000 max-swap-rate=250 swap-timeout=350