[squid-users] Rock store status

[Alex Rousskov]

On 09/13/2016 05:01 AM, FredB wrote:
> One thing, squid restart is very slow because of time required to rebuild the cache
> 
> 2016/09/13 00:25:34|   Took 1498.42 seconds (3972.24 objects/sec). -> Rock
> 2016/09/13 00:00:51|   Took 5.71 seconds (533481.90 objects/sec). -> Diskd

This is a known problem, with several important wrinkles, including:

* Squid start itself is not slow. Cache index build is slow.

* Squid can serve requests, including cache hits while it builds rock
index, but indexing does affect overall Squid performance and hit ratios.

* Avoid comparing loading a "few" ufs entries (from the clean swap
state) with scanning all available cache slots for rock. The biggest
difference is observed for a virtually empty ufs cache that was in use
for a short time (small swap.state). Rock focus is on Squid running for
a long time with a full cache (the common and intended use case).

* We are essentially comparing a from-scratch index build for rock with
a clean index loading for ufs. If you remove all swap state files, ufs
indexing time will probably be worse than that of rock. If you leave
dirty swap state files, then ufs indexing may slow down significantly;
this happens after Squid crashes, for example. Rock indexing does not
depend on the previous Squid state.

* Rock indexing can be optimized in various ways, of course. Many trade
offs are involved, and some optimizations may hurt runtime performance.
For example, there is a trade-off between
    - maintaining a disk index (i.e., swap state files) at runtime
     (and then saving a clean index at shutdown) like UFS stores do and
    - building an index from scratch by scanning the entire cache at
      start like rock stores do.


This is now documented at

http://wiki.squid-cache.org/Features/LargeRockStore#Slow_cache_index_build


HTH,

Alex.