[squid-users] squid won't return cached even with refresh_pattern extra options override-lastmod override-expire ignore-reload ignore-no-store ignore-private store-stale

Alex Rousskov rousskov at measurement-factory.com
Wed Mar 24 19:11:45 UTC 2021


On 3/24/21 2:49 PM, Miroslaw Malinowski wrote:

> looking at the code and reading carefully your response, you're saying
> there is no way you can do it with squid.

With Squid, your options include:

1. Squid source code changes. Should not be too difficult and, IMO, a
high-quality implementation would deserve official acceptance because it
is a generally useful feature in line with existing control knobs.
https://wiki.squid-cache.org/SquidFaq/AboutSquid#How_to_add_a_new_Squid_feature.2C_enhance.2C_of_fix_something.3F

2. An adaptation service that removes Cache-Control:no-cache from the
response before Squid processes it:
https://wiki.squid-cache.org/SquidFaq/ContentAdaptation


HTH,

Alex.

> On Wed, Mar 24, 2021 at 6:28 PM Miroslaw Malinowski wrote:
> 
>     Hi,
> 
>     You've right yes it's revalidating as API server I'm requesting data
>     is setting Cache-Control: no-cache. My question is how I can force
>     squid to cache and not validate as I know it's safe to do so. As
>     I've explained earlier we are making the same request and receiving
>     the same response from 100+ server so as to reduce number of
>     requests to the external server we would like squid to cache the
>     response and issue a cached version.
> 
>     2021/03/24 18:00:54.867 kid1| 22,3| refresh.cc(351) refreshCheck:
>     YES: Must revalidate stale object (origin set no-cache or private)
> 
>     Mirek
> 
>     On Wed, Mar 24, 2021 at 6:15 PM Alex Rousskov
>     <rousskov at measurement-factory.com
>     <mailto:rousskov at measurement-factory.com>> wrote:
> 
>         On 3/24/21 12:48 PM, Miroslaw Malinowski wrote:
> 
>         > Probably, me missing on something silly or it can't be done
>         but I don't
>         > know why but squid won't return the cached version even when I
>         turn all
>         > override options ON in refresh_pattern.
> 
>         AFAICT, no configuration options that can disable revalidation of
>         Cache-Control:no-cache responses. refresh_pattern does not have an
>         (equivalent of) "ignore-no-cache-in-responses" option.
> 
>         IIRC, older Squids were violating an HTTP MUST by forgetting to
>         revalidate Cache-Control:no-cache responses, but that was fixed
>         in [1].
>         Your Squid version has that fix.
> 
>         [1]
>         https://github.com/squid-cache/squid/commit/fa83b766a208b27abed8da4c9073cf8784cf10fa
>         <https://github.com/squid-cache/squid/commit/fa83b766a208b27abed8da4c9073cf8784cf10fa>
> 
> 
>         > With debug, I can see the rule is matched and the cache is
>         fresh but
>         > still in access.log is TCP_REFRESH_MODIFIED
> 
>         > 2021-03-24T15:04:34   squid   .710 kid1| 11,3| http.cc(982)
>         > haveParsedReplyHeaders: decided: cache positively and share
>         because
> 
>         FYI: You are looking at cache.log lines logged _after_ Squid has
>         already
>         decided to refresh the cached version. If you want to analyze
>         why Squid
>         decided to refresh the cached version, you should look _before_
>         Squid
>         logged the request to the server (and before any FwdState.cc
>         lines). I
>         have not checked the details, but I bet that your Squid revalidates
>         because of Cache-Control:no-cache in the response. Look for
>         "YES: Must
>         revalidate stale object".
> 
> 
>         HTH,
> 
>         Alex.
> 
>         > squid conf:
>         > refresh_pattern -i <URL> 4320 80% 129600 override-lastmod
>         > override-expire ignore-reload ignore-no-store ignore-private
>         store-stale
>         >
>         > curl headers:
>         > curl --insecure --verbose --request GET --url 'URL' >/dev/null
>         > * TCP_NODELAY set
>         > * ALPN, offering h2
>         > * ALPN, offering http/1.1
>         > * successfully set certificate verify locations:
>         > *   CAfile: /etc/ssl/certs/ca-certificates.crt
>         >  CApath: /etc/ssl/certs
>         > } [5 bytes data]
>         > * TLSv1.3 (OUT), TLS handshake, Client hello (1):
>         > } [512 bytes data]
>         > * TLSv1.3 (IN), TLS handshake, Server hello (2):
>         > { [122 bytes data]
>         > * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
>         > { [6 bytes data]
>         > * TLSv1.3 (IN), TLS handshake, Certificate (11):
>         > { [1956 bytes data]
>         > * TLSv1.3 (IN), TLS handshake, CERT verify (15):
>         > { [78 bytes data]
>         > * TLSv1.3 (IN), TLS handshake, Finished (20):
>         > { [52 bytes data]
>         > * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
>         > } [1 bytes data]
>         > * TLSv1.3 (OUT), TLS handshake, Finished (20):
>         > } [52 bytes data]
>         > * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
>         >
>         >> GET URL HTTP/1.1
>         >> Host: URL
>         >> User-Agent: curl/7.68.0
>         >> Accept: */*
>         >>
>         > { [5 bytes data]
>         > * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
>         > { [217 bytes data]
>         > * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
>         > { [217 bytes data]
>         > * old SSL session ID is stale, removing
>         > { [5 bytes data]
>         > * Mark bundle as not supporting multiuse
>         > < HTTP/1.1 200 OK
>         > < Cache-Control: no-cache
>         > < Content-Type: application/json
>         > < X-Cloud-Trace-Context: d3c27833b8b4312ce31a2dbae7e12fd0
>         > < Date: Wed, 24 Mar 2021 15:04:34 GMT
>         > < Server: Google Frontend
>         > < Content-Length: 7950
>         > < X-Cache: MISS from server
>         > < X-Cache-Lookup: HIT from server
>         > < Via: 1.1 server (squid/4.14)
>         > < Connection: keep-alive
>         >
>         > access log:
>         > 243 172.16.230.249 TCP_REFRESH_MODIFIED/200 8328 GET URL -
>         > ORIGINAL_DST/IP application/json
>         >
>         > cache log:
>         > 2021-03-24T15:04:34   squid   .710 kid1| 11,3| http.cc(982)
>         > haveParsedReplyHeaders: decided: cache positively and share
>         because
>         > refresh check returned cacheable; HTTP status 200
>         e:=p2V/0x34868914670*3       
>         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3| refresh.cc(470)
>         refreshCheck:
>         > returning FRESH_MIN_RULE       
>         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3| refresh.cc(455)
>         refreshCheck:
>         > Object isn't stale..   
>         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3| refresh.cc(327)
>         refreshCheck:
>         > Staleness = -1         
>         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3| refresh.cc(199)
>         > refreshStaleness: FRESH: age (60 sec) is less than configured
>         minimum
>         > (259200 sec)   
>         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3| refresh.cc(166)
>         > refreshStaleness: No explicit expiry given, using heuristics to
>         > determine freshness    
>         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3| refresh.cc(307)
>         refreshCheck:
>         > entry->timestamp: Wed, 24 Mar 2021 15:04:34 GMT        
>         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3| refresh.cc(305)
>         refreshCheck:
>         > check_time: Wed, 24 Mar 2021 15:05:34 GMT      
>         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3| refresh.cc(303)
>         refreshCheck:
>         > age: 60        
>         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3| refresh.cc(301)
>         refreshCheck:
>         > Matched 'URL 259200 80%% 7776000'      
>         > 2021-03-24T15:04:34   squid   .710 kid1| 22,3| refresh.cc(279)
>         refreshCheck:
>         > checking freshness of URI: https://URL <https://URL>
>         <https://URL <https://URL>>
>         >
>         >
>         > _______________________________________________
>         > squid-users mailing list
>         > squid-users at lists.squid-cache.org
>         <mailto:squid-users at lists.squid-cache.org>
>         > http://lists.squid-cache.org/listinfo/squid-users
>         <http://lists.squid-cache.org/listinfo/squid-users>
>         >
> 



More information about the squid-users mailing list