[squid-users] squid stores multiple copies of identical ETags

[Tabacchiera, Stefano]

Hello there,
I have the following issue with squid (squid-3.5.20-12.el7_6.1.x86_64  on RHEL 7.6)

A client requests a json with "no-cache" header via proxy.
Squid forwards the request to origin server, which replies with "ETag" header (object is cachable).
Squid stores the object in cache_dir and forwards back to the client.

The client is pushing same request at high rate (~10/sec), regardless its cachable status.
Squid keeps forwarding the request and - here is my issue - keeps storing the same identical object on disk.
I have thousands of copies of the same Etag on disk.

Is there a way to avoid this? I think Squid should store a single copy per-URL/ETAg of the object.
I'd like to avoid an ad-hoc reload-into-ims refresh-pattern.

Here's an example of headers sequence:



  1.  CLIENT --> SQUID
GET http://xxx.xxx.xxx.xxx/blah/FEED.json HTTP/1.1
Accept-Encoding: gzip
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2
Content-Language: en-US
Cache-Control: no-cache
Pragma: no-cache
Host: xxx.xxx.xxx.xxx
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Proxy-Connection: keep-alive


  1.  SQUID --> ORIGIN SERVER
GET /blah/FEED.json HTTP/1.1
Accept-Encoding: gzip
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2
Content-Language: en-US
Pragma: no-cache
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Host: xxx.xxx.xxx.xxx
Via: 1.1 RP-PRXSQUID-2 (squid/3.5.20)
X-Forwarded-For: unknown
Cache-Control: no-cache
Connection: keep-alive


  1.  ORIGIN SERVER --> SQUID
HTTP/1.1 200 OK
Server: Apache/2.2.16 (Debian)
Last-Modified: Fri, 26 Jun 2020 11:46:00 GMT
ETag: "62b905-a7cfa-5a8fb40bdee00"
Accept-Ranges: bytes
Content-Length: 687354
Keep-Alive: timeout=4, max=45
Connection: Keep-Alive
Content-Type: text/plain


  1.  SQUID --> CLIENT
HTTP/1.1 200 OK
Date: Fri, 26 Jun 2020 11:52:15 GMT
Server: Apache/2.2.16 (Debian)
Last-Modified: Fri, 26 Jun 2020 11:46:00 GMT
ETag: "62b905-a7cfa-5a8fb40bdee00"
Accept-Ranges: bytes
Content-Length: 687354
Content-Type: text/plain
X-Cache: MISS from RP-PRXSQUID-2
X-Cache-Lookup: HIT from RP-PRXSQUID-2:3128
Via: 1.1 RP-PRXSQUID-2 (squid/3.5.20)
Connection: keep-alive

Thanks
ST
____________________________________________________________________________________ La presente comunicazione ed i suoi allegati e' destinata esclusivamente ai destinatari. Qualsiasi suo utilizzo, comunicazione o diffusione non autorizzata e' proibita. Se ha ricevuto questa comunicazione per errore, la preghiamo di darne immediata comunicazione al mittente e di cancellare tutte le informazioni erroneamente acquisite. Grazie This message and its attachments are intended only for use by the addressees. Any use, re-transmission or dissemination not authorized of it is prohibited. If you received this e-mail in error, please inform the sender immediately and delete all the material. Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20200626/1d472686/attachment.html>