[squid-users] Cache digest vs ICP

Mon Oct 2 14:28:36 UTC 2017

Alex, thank you for your response!

2017-09-27 18:06 GMT+03:00 Alex Rousskov <rousskov at measurement-factory.com>:

> On 09/27/2017 03:46 AM, Veiko Kukk wrote:
>
> > Siblings are configured with no-proxy keyword to achieve that they don't
> > cache what other siblings already have in their cache.
>
> I assume that by "no-proxy" you meant "proxy-only".
>
> True, that was my mistake.

>
> > This is to minimize data usage costs from origin servers.
>
> The proxy-only option does not minimize the amount of data transmitted
> between a proxy and the origin server. It reduces cache duplication
> among cache peers.

Exactly.

>
> > So far digest_generation has been set to off and only ICP has been used
> > between siblings. Mostly because digest stats had shown many rejects
> > (not containing 100% of cache objects) and documentation about digests
> > is confusing up to statements that while rebuilding digest, squid will
> > stop serving requests.
>
> Please point me to the location of that statement. IMHO, it is not
> confusing but incorrect.

I found it in the book by Duane Wessels
http://etutorials.org/Server+Administration/Squid.+The+definitive+guide/Chapter+10.+Talking+to+Other+Squids/10.7+Cache+Digests/
Quoting: During each invocation of the rebuild function, Squid adds some
percentage of the cache to the digest. Squid doesn't process user requests
while this function runs.

>
> Cache Digests are not SMP aware (but should be). You may be able to work
> around that limitation using SMP macros, but I have not tested that. I
> do not remember whether a worker that is not configured to generate a
> digest will still look it up in the cache when a peer asks for it.
> Hopefully, the worker will do that lookup.
>
> That sounds very interesting. Could you point me to sample configuration?

>
> > Digest
> > documentation states that it's including based on refresh_pattern. It's
> > a problem because to get squid working as we want, we had to use
> > offline_mode on.
>
> If Cache Digests do not honor offline_mode, it is a (staleness
> estimation code) bug that should be reported and fixed.
>

Can't confirm this now, but we had issues with that earlier.
http://squid-web-proxy-cache.1019090.n4.nabble.com/Never-expire-any-object-Squid-configuration-td4677160.html

>
> > * Why does sibling false positive result in sending client 504 and not
> > trying next sibling or parent? CD_SIBLING_HIT/192.168.1.52
> > TCP_MISS/504. How to achieve proceeding with next cache_peer?
>
> Sounds like bug #4223 to me:
> http://bugs.squid-cache.org/show_bug.cgi?id=4223
>
> I've patched 3.5.27 with patch found under that bug and build rpm for
testing, and so far have not encountered that error anymore.

I have another issue. How frequently are cache digests refreshed from
siblings? It seems to me that it takes quite a lot time and i have not
found anything in documentation that could help enfroce digest refreshing.
In test system, i've set 'digest_rebuild_period 60 second'. With clean
cache and running test downloads sibling1 very quickly updates it's cache
digest:

Local Digest:
store digest: size: 10492 bytes
entries: count: 415 capacity: 16787 util: 2%
deletion attempts: 0
bits: per entry: 5 on: 1648 capacity: 83936 util: 2%
bit-seq: count: 3224 avg.len: 26.03
added: 415 rejected: 0 ( 0.00 %) del-ed: 0
collisions: on add: 0.00 % on rej: -1.00 %

I've waited at least 20 minutes, several times ran downloads agains
sibling2 (clean cache too) and sibling2 (192.168.1.52) still shows old,
almost empty cache digest for sibling1(192.168.1.51):

Peer Digests:
no guess stats for all peers available

Per-peer statistics:

peer digest from 192.168.1.51
no guess stats for 192.168.1.51 available

event timestamp secs from now secs from init
initialized 1506952649 -1602 +0
needed 1506953341 -910 +692
requested 1506953341 -910 +692
received 1506953341 -910 +692
next_check 1506956584 +2333 +3935
peer digest state:
needed: yes, usable: yes, requested:  no

last retry delay: 0 secs
last request response time: 0 secs
last request result: success

peer digest traffic:
requests sent: 1, volume: 0 KB
replies recv:  1, volume: 0 KB

peer digest structure:
192.168.1.51 digest: size: 32 bytes
entries: count: 51 capacity: 51 util: 100%
deletion attempts: 0
bits: per entry: 5 on: 142 capacity: 256 util: 55%
bit-seq: count: 131 avg.len: 1.95

Algorithm usage:
Cache Digest:       0 ( -1%)
Icp:                0 ( -1%)
Total:              0 ( -1%)

Local Digest:
store digest: size: 1461 bytes
entries: count: 75 capacity: 2337 util: 3%
deletion attempts: 0
bits: per entry: 5 on: 296 capacity: 11688 util: 3%
bit-seq: count: 572 avg.len: 20.43
added: 75 rejected: 0 ( 0.00 %) del-ed: 0
collisions: on add: 0.00 % on rej: -1.00 %

Why is it so?
How is cache digest from siblings refreshed?
How can sibling cache digest refreshed more frequently?

Best regards,
Veiko
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20171002/47fd8c78/attachment.html>