[squid-users] R: squid stores multiple copies of identical ETags

Tue Jun 30 09:10:57 UTC 2020

>> I mean: squid would store a new copy of the object while leaving the

>> old copy deletion to cleanup task?

>Some parts of the cleanup process may be delegated. The details depend on the cache_dir type. I do not know or remember aufs specifics, but I suspect that all ufs-based cache_dirs, including aufs, use lazy garbage collection >(under normal circumstances). The cache_swap_low and cache_swap_high directives should determine what is "normal".

Alex, you were absolutely right.

I managed to reproduce the case.

On test environment I set up "cache_swap_low 1" and "cache_swap_low 2" and enabled the store_log.

Then I tailed the store_log and watched the evolution of cache_dir, while running squidclient toward origin server every 100ms.

Store.log:

1593504359.704 SWAPOUT 00 000098C5 1865A3A26D411E7C0D8D87770720E405  200 1593504359 1593504061        -1 text/plain 544275/544275 GET http://xxx.xxx.xxx.xxx/blah/FEED.json

1593504359.858 SWAPOUT 00 000098C6 1865A3A26D411E7C0D8D87770720E405  200 1593504359 1593504061        -1 text/plain 544275/544275 GET http://xxx.xxx.xxx.xxx/blah/FEED.json

1593504360.015 SWAPOUT 00 000098C7 1865A3A26D411E7C0D8D87770720E405  200 1593504359 1593504061        -1 text/plain 544275/544275 GET http://xxx.xxx.xxx.xxx/blah/FEED.json

1593504360.170 SWAPOUT 00 000098C8 1865A3A26D411E7C0D8D87770720E405  200 1593504359 1593504061        -1 text/plain 544275/544275 GET http://xxx.xxx.xxx.xxx/blah/FEED.json

1593504360.324 SWAPOUT 00 000098C9 1865A3A26D411E7C0D8D87770720E405  200 1593504359 1593504061        -1 text/plain 544275/544275 GET http://xxx.xxx.xxx.xxx/blah/FEED.json

1593504360.476 SWAPOUT 00 000098CA 1865A3A26D411E7C0D8D87770720E405  200 1593504359 1593504061        -1 text/plain 544275/544275 GET http://xxx.xxx.xxx.xxx/blah/FEED.json

1593504360.634 SWAPOUT 00 000098CB 1865A3A26D411E7C0D8D87770720E405  200 1593504360 1593504061        -1 text/plain 544275/544275 GET http://xxx.xxx.xxx.xxx/blah/FEED.json

1593504360.788 SWAPOUT 00 000098CC 1865A3A26D411E7C0D8D87770720E405  200 1593504360 1593504061        -1 text/plain 544275/544275 GET http://xxx.xxx.xxx.xxx/blah/FEED.json

1593504360.941 SWAPOUT 00 000098CD 1865A3A26D411E7C0D8D87770720E405  200 1593504360 1593504061        -1 text/plain 544275/544275 GET http://xxx.xxx.xxx.xxx/blah/FEED.json

1593504361.096 SWAPOUT 00 000098CE 1865A3A26D411E7C0D8D87770720E405  200 1593504360 1593504061        -1 text/plain 544275/544275 GET http://xxx.xxx.xxx.xxx/blah/FEED.json

1593504361.249 SWAPOUT 00 000098CF 1865A3A26D411E7C0D8D87770720E405  200 1593504360 1593504061        -1 text/plain 544275/544275 GET http://xxx.xxx.xxx.xxx/blah/FEED.json

1593504361.403 SWAPOUT 00 000098D0 1865A3A26D411E7C0D8D87770720E405  200 1593504360 1593504061        -1 text/plain 544275/544275 GET http://xxx.xxx.xxx.xxx/blah/FEED.json

1593504361.556 SWAPOUT 00 000098D1 1865A3A26D411E7C0D8D87770720E405  200 1593504360 1593504061        -1 text/plain 544275/544275 GET http://xxx.xxx.xxx.xxx/blah/FEED.json

1593504361.607 RELEASE 00 000098D1 1865A3A26D411E7C0D8D87770720E405   ?         ?         ?         ? ?/? ?/? ? ?

The cached object was gradually appearing in cache_dir, until the “RELEASE” line showed up in the store.log.

At this right moment, all copies of the object stored on disk were deleted.

So I’m assuming that only one object on disk (the last one retrieved) is the object referenced as “active” by squid, all the rest being trashable.

Since the client is forcing a “no-cache” header, squid does what the client is asking for, and every time it stores the object on disk.

I’m also assuming that IF another client asked the same object without the “no-cache” header, squid would serve the latest cached object on disk.

If I’m right so far, squid never “overwrites” the old copy of an object on disk. Instead, it stores a new one, marking it as “active”, and let the deletion task to (a)ufs threads.

Could this this way?

Thanks!

ST

____________________________________________________________________________________ La presente comunicazione ed i suoi allegati e' destinata esclusivamente ai destinatari. Qualsiasi suo utilizzo, comunicazione o diffusione non autorizzata e' proibita. Se ha ricevuto questa comunicazione per errore, la preghiamo di darne immediata comunicazione al mittente e di cancellare tutte le informazioni erroneamente acquisite. Grazie This message and its attachments are intended only for use by the addressees. Any use, re-transmission or dissemination not authorized of it is prohibited. If you received this e-mail in error, please inform the sender immediately and delete all the material. Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20200630/b46ce9c0/attachment.html>