[squid-users] squid stores multiple copies of identical ETags

[Alex Rousskov]

On 6/28/20 2:41 PM, Tabacchiera, Stefano wrote:

>>    Squid does not know that the response headers and body have not changed.
>>    Squid could, in theory, trust the URL+Vary+ETag+etc. combination as a
>>    precise response identity, but it is a bit risky to do that by default
>>    because ETags/etc. might lie. There is currently no code implementing
>>    that optimization either.

> now I know that only URL/Vary are taken into account when storing an object.

For the record: I did not say or imply that only URL/Vary are taken into
account. Other request aspects such as request method also influence the
cache key.


> I mean: squid would store a new copy of the object while leaving the
> old copy deletion to cleanup task?

Some parts of the cleanup process may be delegated. The details depend
on the cache_dir type. I do not know or remember aufs specifics, but I
suspect that all ufs-based cache_dirs, including aufs, use lazy garbage
collection (under normal circumstances). The cache_swap_low and
cache_swap_high directives should determine what is "normal".


> I'm hesitant to to turn the store_log on, 'cause the performance impact.
> Btw, is there a specific debug_option?

FWIW, I would not try to debug this on a live/production cache,
especially if you are not used to navigating debugging cache.logs from
busy proxies. The same bug (if any) should be reproducible in lab settings.

These debugging sections may be relevant (but this is not meant as a
comprehensive list, and I do not know what exactly you need to look at):

> doc/debug-sections.txt:section 20    Memory Cache
> doc/debug-sections.txt:section 20    Storage Manager MD5 Cache Keys
> doc/debug-sections.txt:section 20    Store Controller
> doc/debug-sections.txt:section 32    Asynchronous Disk I/O
> doc/debug-sections.txt:section 43    AIOPS
> doc/debug-sections.txt:section 47    Store Directory Routines
> doc/debug-sections.txt:section 81    Store HEAP Removal Policies

Most of the AUFS/AIO code is in src/DiskIO/DiskThreads


HTH,

Alex.