[squid-users] Squid transparent not caching apt requests from deb.debian.org

Amos Jeffries squid3 at treenet.co.nz
Sat Apr 4 15:18:23 UTC 2020


On 5/04/20 2:53 am, Alex Rousskov wrote:
> On 4/3/20 4:55 PM, zrm wrote:
>> On 4/3/20 16:34, Alex Rousskov wrote:
>>> On 4/3/20 4:26 PM, zrm wrote:
>>>> In the first case we get TCP_MISS every time because it isn't caching
>>>> the data, in the second case it's only the first time and after that we
>>>> get TCP_REFRESH_UNMODIFIED. But how and why is this happening?
> 
>> squid to apt:
>> ---------
>> X-Cache: MISS from tproxy
>> X-Cache-Lookup: MISS from tproxy:3130
> 
>> squid to wget:
>> ---------
>> X-Cache: MISS from tproxy
>> X-Cache-Lookup: MISS from tproxy:3130
> 
> The headers you have posted tell us that the object was not in Squid
> cache when apt and wget transactions started. Since wget was started
> after apt, we can speculate that apt transaction did not cache the
> object. This is consistent with your observations. I saw nothing in the
> posted headers that would explain the difference between apt and wget
> outcomes.
> 
> Unfortunately, you did not show the headers for the case where the
> object actually got cached by Squid. You can probably do that by
> repeating the wget transaction twice. I would also repeat the apt
> transaction twice after that. It would also be interesting to see the
> wget-apt and apt-wget sequences. In summary, I would do
> wget-wget-apt-apt-wget-wget. Sleep for 10+ seconds between each
> transaction to reduce chances of overlapping cache operations.
> 
> BTW, you probably do not need to make ALL,2 logs pretty -- we can figure
> out what happens based on Squid messages if you submit one transaction
> at a time and disclose transaction sequence. You can just post (a link
> to) raw (or sanitized) logs. Compress them if they are too big.
> 
> Before sharing the logs, please double check that the problem you want
> to address was reproduced during the test.
> 
> 
>> Last-Modified: Sat, 15 Jun 2019 17:46:35 GMT
>> ETag: "1389dc-58b605823fa6e"
>> Cache-Control: public, max-age=2592000
>> Content-Length: 1280476
>> Age: 4248100
> 
> FWIW: The object is 4'248'100 seconds old. The object max-age is
> 2'592'000 seconds. Your Squid is probably using an internal max-age of
> 259'200 seconds, so Squid will require cache hit revalidation during
> subsequent transactions after Squid caches the object (if it caches it).
> 
> 

One thing to notice as well is that the object delivered by the upstream
caches expired over 19 days before these tests took place:

> Cache-Control: public, max-age=2592000
> Age: 4248100

The request from Squid in both transactions was to receive content no
longer than 3 days old. The other caches ignored that requirement and
served old content from their storage, apparently without even checking
an origin whether it was stale.

Amos


More information about the squid-users mailing list