<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">Alex, thank you for your response!</div><div class="gmail_quote"><br></div><div class="gmail_quote">2017-09-27 18:06 GMT+03:00 Alex Rousskov <span dir="ltr"><<a href="mailto:rousskov@measurement-factory.com" target="_blank">rousskov@measurement-factory.<wbr>com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="gmail-m_6333848106523998604gmail-">On 09/27/2017 03:46 AM, Veiko Kukk wrote:<br>
<br>
> Siblings are configured with no-proxy keyword to achieve that they don't<br>
> cache what other siblings already have in their cache.<br>
<br>
</span>I assume that by "no-proxy" you meant "proxy-only".<br>
<span class="gmail-m_6333848106523998604gmail-"><br></span></blockquote><div>True, that was my mistake. </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="gmail-m_6333848106523998604gmail-">
<br>
> This is to minimize data usage costs from origin servers.<br>
<br>
</span>The proxy-only option does not minimize the amount of data transmitted<br>
between a proxy and the origin server. It reduces cache duplication<br>
among cache peers.</blockquote><div> </div><div>Exactly. <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="gmail-m_6333848106523998604gmail-"><br>
> So far digest_generation has been set to off and only ICP has been used<br>
> between siblings. Mostly because digest stats had shown many rejects<br>
> (not containing 100% of cache objects) and documentation about digests<br>
> is confusing up to statements that while rebuilding digest, squid will<br>
> stop serving requests.<br>
<br>
</span>Please point me to the location of that statement. IMHO, it is not<br>
confusing but incorrect.</blockquote><div><br></div>I found it in the book by Duane Wessels <a href="http://etutorials.org/Server+Administration/Squid.+The+definitive+guide/Chapter+10.+Talking+to+Other+Squids/10.7+Cache+Digests/">http://etutorials.org/Server+Administration/Squid.+The+definitive+guide/Chapter+10.+Talking+to+Other+Squids/10.7+Cache+Digests/</a><br>Quoting: During each invocation of the rebuild function, Squid adds some percentage of the cache to the digest. Squid doesn't process user requests while this function runs.<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
Cache Digests are not SMP aware (but should be). You may be able to work<br>
around that limitation using SMP macros, but I have not tested that. I<br>
do not remember whether a worker that is not configured to generate a<br>
digest will still look it up in the cache when a peer asks for it.<br>
Hopefully, the worker will do that lookup.<br>
<span class="gmail-m_6333848106523998604gmail-"><br></span></blockquote><div>That sounds very interesting. Could you point me to sample configuration?</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="gmail-m_6333848106523998604gmail-">
<br>
> Digest<br>
> documentation states that it's including based on refresh_pattern. It's<br>
> a problem because to get squid working as we want, we had to use<br>
> offline_mode on.<br>
<br>
</span>If Cache Digests do not honor offline_mode, it is a (staleness<br>
estimation code) bug that should be reported and fixed.<br></blockquote><div><br></div><div>Can't confirm this now, but we had issues with that earlier. <a href="http://squid-web-proxy-cache.1019090.n4.nabble.com/Never-expire-any-object-Squid-configuration-td4677160.html">http://squid-web-proxy-cache.1019090.n4.nabble.com/Never-expire-any-object-Squid-configuration-td4677160.html</a> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="gmail-m_6333848106523998604gmail-"><br></span><span class="gmail-m_6333848106523998604gmail-">> * Why does sibling false positive result in sending client 504 and not<br>
> trying next sibling or parent? CD_SIBLING_HIT/<a href="http://192.168.1.52" rel="noreferrer" target="_blank">192.168<wbr>.1.52</a><br>
> TCP_MISS/504. How to achieve proceeding with next cache_peer?<br>
<br>
</span>Sounds like bug #4223 to me:<br>
<a href="http://bugs.squid-cache.org/show_bug.cgi?id=4223" rel="noreferrer" target="_blank">http://bugs.squid-cache.org/sh<wbr>ow_bug.cgi?id=4223</a><br>
<br></blockquote><div>I've patched 3.5.27 with patch found under that bug and build rpm for testing, and so far have not encountered that error anymore.</div><div><br></div><div>I have another issue. How frequently are cache digests refreshed from siblings? It seems to me that it takes quite a lot time and i have not found anything in documentation that could help enfroce digest refreshing. In test system, i've set 'digest_rebuild_period 60 second'. With clean cache and running test downloads sibling1 very quickly updates it's cache digest:</div><div><br></div><div><div>Local Digest:</div><div>store digest: size: 10492 bytes</div><div><span style="white-space:pre"> </span> entries: count: 415 capacity: 16787 util: 2%</div><div><span style="white-space:pre"> </span> deletion attempts: 0</div><div><span style="white-space:pre"> </span> bits: per entry: 5 on: 1648 capacity: 83936 util: 2%</div><div><span style="white-space:pre"> </span> bit-seq: count: 3224 avg.len: 26.03</div><div><span style="white-space:pre"> </span> added: 415 rejected: 0 ( 0.00 %) del-ed: 0</div><div><span style="white-space:pre"> </span> collisions: on add: 0.00 % on rej: -1.00 %</div></div><div><br></div><div>I've waited at least 20 minutes, several times ran downloads agains sibling2 (clean cache too) and sibling2 (192.168.1.52) still shows old, almost empty cache digest for sibling1(192.168.1.51):</div><div><br></div><div><div>Peer Digests:</div><div>no guess stats for all peers available</div><div><br></div><div>Per-peer statistics:</div><div><br></div><div>peer digest from 192.168.1.51</div><div>no guess stats for 192.168.1.51 available</div><div><br></div><div>event<span style="white-space:pre"> </span> timestamp<span style="white-space:pre"> </span> secs from now<span style="white-space:pre"> </span> secs from init</div><div>initialized<span style="white-space:pre"> </span> 1506952649<span style="white-space:pre"> </span> -1602<span style="white-space:pre"> </span> +0</div><div>needed<span style="white-space:pre"> </span> 1506953341<span style="white-space:pre"> </span> -910<span style="white-space:pre"> </span> +692</div><div>requested<span style="white-space:pre"> </span> 1506953341<span style="white-space:pre"> </span> -910<span style="white-space:pre"> </span> +692</div><div>received<span style="white-space:pre"> </span> 1506953341<span style="white-space:pre"> </span> -910<span style="white-space:pre"> </span> +692</div><div>next_check<span style="white-space:pre"> </span> 1506956584<span style="white-space:pre"> </span> +2333<span style="white-space:pre"> </span> +3935</div><div>peer digest state:</div><div><span style="white-space:pre"> </span>needed: yes, usable: yes, requested: no</div><div><br></div><div><span style="white-space:pre"> </span>last retry delay: 0 secs</div><div><span style="white-space:pre"> </span>last request response time: 0 secs</div><div><span style="white-space:pre"> </span>last request result: success</div><div><br></div><div>peer digest traffic:</div><div><span style="white-space:pre"> </span>requests sent: 1, volume: 0 KB</div><div><span style="white-space:pre"> </span>replies recv: 1, volume: 0 KB</div><div><br></div><div>peer digest structure:</div><div>192.168.1.51 digest: size: 32 bytes</div><div><span style="white-space:pre"> </span> entries: count: 51 capacity: 51 util: 100%</div><div><span style="white-space:pre"> </span> deletion attempts: 0</div><div><span style="white-space:pre"> </span> bits: per entry: 5 on: 142 capacity: 256 util: 55%</div><div><span style="white-space:pre"> </span> bit-seq: count: 131 avg.len: 1.95</div><div><br></div><div><br></div><div>Algorithm usage:</div><div>Cache Digest: 0 ( -1%)</div><div>Icp: 0 ( -1%)</div><div>Total: 0 ( -1%)</div><div><br></div><div>Local Digest:</div><div>store digest: size: 1461 bytes</div><div><span style="white-space:pre"> </span> entries: count: 75 capacity: 2337 util: 3%</div><div><span style="white-space:pre"> </span> deletion attempts: 0</div><div><span style="white-space:pre"> </span> bits: per entry: 5 on: 296 capacity: 11688 util: 3%</div><div><span style="white-space:pre"> </span> bit-seq: count: 572 avg.len: 20.43</div><div><span style="white-space:pre"> </span> added: 75 rejected: 0 ( 0.00 %) del-ed: 0</div><div><span style="white-space:pre"> </span> collisions: on add: 0.00 % on rej: -1.00 %</div></div><div><br></div><div>Why is it so?</div><div>How is cache digest from siblings refreshed?</div><div>How can sibling cache digest refreshed more frequently?</div><div><br></div><div>Best regards,</div><div>Veiko</div></div></div></div>