<div dir="ltr"><div>I thought about upper service but as is not required at the moment, introducing extra hop just to remove the header looks a bit like a hammer approach. I'll look into how easily I can amend the code as the other option is to introduce a proxy like a feature to the application, so either way, it is a code change. The only problem here is that it's an OPNSense squid service so I have to compile from source on BSD and then keep adding in manually each time they do the update.</div><div><br></div><div>Mirek<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Mar 24, 2021 at 7:11 PM Alex Rousskov <<a href="mailto:rousskov@measurement-factory.com">rousskov@measurement-factory.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 3/24/21 2:49 PM, Miroslaw Malinowski wrote:<br>
<br>
> looking at the code and reading carefully your response, you're saying<br>
> there is no way you can do it with squid.<br>
<br>
With Squid, your options include:<br>
<br>
1. Squid source code changes. Should not be too difficult and, IMO, a<br>
high-quality implementation would deserve official acceptance because it<br>
is a generally useful feature in line with existing control knobs.<br>
<a href="https://wiki.squid-cache.org/SquidFaq/AboutSquid#How_to_add_a_new_Squid_feature.2C_enhance.2C_of_fix_something.3F" rel="noreferrer" target="_blank">https://wiki.squid-cache.org/SquidFaq/AboutSquid#How_to_add_a_new_Squid_feature.2C_enhance.2C_of_fix_something.3F</a><br>
<br>
2. An adaptation service that removes Cache-Control:no-cache from the<br>
response before Squid processes it:<br>
<a href="https://wiki.squid-cache.org/SquidFaq/ContentAdaptation" rel="noreferrer" target="_blank">https://wiki.squid-cache.org/SquidFaq/ContentAdaptation</a><br>
<br>
<br>
HTH,<br>
<br>
Alex.<br>
<br>
> On Wed, Mar 24, 2021 at 6:28 PM Miroslaw Malinowski wrote:<br>
> <br>
> Hi,<br>
> <br>
> You've right yes it's revalidating as API server I'm requesting data<br>
> is setting Cache-Control: no-cache. My question is how I can force<br>
> squid to cache and not validate as I know it's safe to do so. As<br>
> I've explained earlier we are making the same request and receiving<br>
> the same response from 100+ server so as to reduce number of<br>
> requests to the external server we would like squid to cache the<br>
> response and issue a cached version.<br>
> <br>
> 2021/03/24 18:00:54.867 kid1| 22,3| refresh.cc(351) refreshCheck:<br>
> YES: Must revalidate stale object (origin set no-cache or private)<br>
> <br>
> Mirek<br>
> <br>
> On Wed, Mar 24, 2021 at 6:15 PM Alex Rousskov<br>
> <<a href="mailto:rousskov@measurement-factory.com" target="_blank">rousskov@measurement-factory.com</a><br>
> <mailto:<a href="mailto:rousskov@measurement-factory.com" target="_blank">rousskov@measurement-factory.com</a>>> wrote:<br>
> <br>
> On 3/24/21 12:48 PM, Miroslaw Malinowski wrote:<br>
> <br>
> > Probably, me missing on something silly or it can't be done<br>
> but I don't<br>
> > know why but squid won't return the cached version even when I<br>
> turn all<br>
> > override options ON in refresh_pattern.<br>
> <br>
> AFAICT, no configuration options that can disable revalidation of<br>
> Cache-Control:no-cache responses. refresh_pattern does not have an<br>
> (equivalent of) "ignore-no-cache-in-responses" option.<br>
> <br>
> IIRC, older Squids were violating an HTTP MUST by forgetting to<br>
> revalidate Cache-Control:no-cache responses, but that was fixed<br>
> in [1].<br>
> Your Squid version has that fix.<br>
> <br>
> [1]<br>
> <a href="https://github.com/squid-cache/squid/commit/fa83b766a208b27abed8da4c9073cf8784cf10fa" rel="noreferrer" target="_blank">https://github.com/squid-cache/squid/commit/fa83b766a208b27abed8da4c9073cf8784cf10fa</a><br>
> <<a href="https://github.com/squid-cache/squid/commit/fa83b766a208b27abed8da4c9073cf8784cf10fa" rel="noreferrer" target="_blank">https://github.com/squid-cache/squid/commit/fa83b766a208b27abed8da4c9073cf8784cf10fa</a>><br>
> <br>
> <br>
> > With debug, I can see the rule is matched and the cache is<br>
> fresh but<br>
> > still in access.log is TCP_REFRESH_MODIFIED<br>
> <br>
> > 2021-03-24T15:04:34 squid .710 kid1| 11,3| http.cc(982)<br>
> > haveParsedReplyHeaders: decided: cache positively and share<br>
> because<br>
> <br>
> FYI: You are looking at cache.log lines logged _after_ Squid has<br>
> already<br>
> decided to refresh the cached version. If you want to analyze<br>
> why Squid<br>
> decided to refresh the cached version, you should look _before_<br>
> Squid<br>
> logged the request to the server (and before any FwdState.cc<br>
> lines). I<br>
> have not checked the details, but I bet that your Squid revalidates<br>
> because of Cache-Control:no-cache in the response. Look for<br>
> "YES: Must<br>
> revalidate stale object".<br>
> <br>
> <br>
> HTH,<br>
> <br>
> Alex.<br>
> <br>
> > squid conf:<br>
> > refresh_pattern -i <URL> 4320 80% 129600 override-lastmod<br>
> > override-expire ignore-reload ignore-no-store ignore-private<br>
> store-stale<br>
> ><br>
> > curl headers:<br>
> > curl --insecure --verbose --request GET --url 'URL' >/dev/null<br>
> > * TCP_NODELAY set<br>
> > * ALPN, offering h2<br>
> > * ALPN, offering http/1.1<br>
> > * successfully set certificate verify locations:<br>
> > * CAfile: /etc/ssl/certs/ca-certificates.crt<br>
> > CApath: /etc/ssl/certs<br>
> > } [5 bytes data]<br>
> > * TLSv1.3 (OUT), TLS handshake, Client hello (1):<br>
> > } [512 bytes data]<br>
> > * TLSv1.3 (IN), TLS handshake, Server hello (2):<br>
> > { [122 bytes data]<br>
> > * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):<br>
> > { [6 bytes data]<br>
> > * TLSv1.3 (IN), TLS handshake, Certificate (11):<br>
> > { [1956 bytes data]<br>
> > * TLSv1.3 (IN), TLS handshake, CERT verify (15):<br>
> > { [78 bytes data]<br>
> > * TLSv1.3 (IN), TLS handshake, Finished (20):<br>
> > { [52 bytes data]<br>
> > * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):<br>
> > } [1 bytes data]<br>
> > * TLSv1.3 (OUT), TLS handshake, Finished (20):<br>
> > } [52 bytes data]<br>
> > * SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384<br>
> ><br>
> >> GET URL HTTP/1.1<br>
> >> Host: URL<br>
> >> User-Agent: curl/7.68.0<br>
> >> Accept: */*<br>
> >><br>
> > { [5 bytes data]<br>
> > * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):<br>
> > { [217 bytes data]<br>
> > * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):<br>
> > { [217 bytes data]<br>
> > * old SSL session ID is stale, removing<br>
> > { [5 bytes data]<br>
> > * Mark bundle as not supporting multiuse<br>
> > < HTTP/1.1 200 OK<br>
> > < Cache-Control: no-cache<br>
> > < Content-Type: application/json<br>
> > < X-Cloud-Trace-Context: d3c27833b8b4312ce31a2dbae7e12fd0<br>
> > < Date: Wed, 24 Mar 2021 15:04:34 GMT<br>
> > < Server: Google Frontend<br>
> > < Content-Length: 7950<br>
> > < X-Cache: MISS from server<br>
> > < X-Cache-Lookup: HIT from server<br>
> > < Via: 1.1 server (squid/4.14)<br>
> > < Connection: keep-alive<br>
> ><br>
> > access log:<br>
> > 243 172.16.230.249 TCP_REFRESH_MODIFIED/200 8328 GET URL -<br>
> > ORIGINAL_DST/IP application/json<br>
> ><br>
> > cache log:<br>
> > 2021-03-24T15:04:34 squid .710 kid1| 11,3| http.cc(982)<br>
> > haveParsedReplyHeaders: decided: cache positively and share<br>
> because<br>
> > refresh check returned cacheable; HTTP status 200<br>
> e:=p2V/0x34868914670*3 <br>
> > 2021-03-24T15:04:34 squid .710 kid1| 22,3| refresh.cc(470)<br>
> refreshCheck:<br>
> > returning FRESH_MIN_RULE <br>
> > 2021-03-24T15:04:34 squid .710 kid1| 22,3| refresh.cc(455)<br>
> refreshCheck:<br>
> > Object isn't stale.. <br>
> > 2021-03-24T15:04:34 squid .710 kid1| 22,3| refresh.cc(327)<br>
> refreshCheck:<br>
> > Staleness = -1 <br>
> > 2021-03-24T15:04:34 squid .710 kid1| 22,3| refresh.cc(199)<br>
> > refreshStaleness: FRESH: age (60 sec) is less than configured<br>
> minimum<br>
> > (259200 sec) <br>
> > 2021-03-24T15:04:34 squid .710 kid1| 22,3| refresh.cc(166)<br>
> > refreshStaleness: No explicit expiry given, using heuristics to<br>
> > determine freshness <br>
> > 2021-03-24T15:04:34 squid .710 kid1| 22,3| refresh.cc(307)<br>
> refreshCheck:<br>
> > entry->timestamp: Wed, 24 Mar 2021 15:04:34 GMT <br>
> > 2021-03-24T15:04:34 squid .710 kid1| 22,3| refresh.cc(305)<br>
> refreshCheck:<br>
> > check_time: Wed, 24 Mar 2021 15:05:34 GMT <br>
> > 2021-03-24T15:04:34 squid .710 kid1| 22,3| refresh.cc(303)<br>
> refreshCheck:<br>
> > age: 60 <br>
> > 2021-03-24T15:04:34 squid .710 kid1| 22,3| refresh.cc(301)<br>
> refreshCheck:<br>
> > Matched 'URL 259200 80%% 7776000' <br>
> > 2021-03-24T15:04:34 squid .710 kid1| 22,3| refresh.cc(279)<br>
> refreshCheck:<br>
> > checking freshness of URI: <a href="https://URL" rel="noreferrer" target="_blank">https://URL</a> <<a href="https://URL" rel="noreferrer" target="_blank">https://URL</a>><br>
> <<a href="https://URL" rel="noreferrer" target="_blank">https://URL</a> <<a href="https://URL" rel="noreferrer" target="_blank">https://URL</a>>><br>
> ><br>
> ><br>
> > _______________________________________________<br>
> > squid-users mailing list<br>
> > <a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a><br>
> <mailto:<a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a>><br>
> > <a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a><br>
> <<a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a>><br>
> ><br>
> <br>
<br>
</blockquote></div>