<div dir="auto">Thanks, Alex.</div><div dir="auto"><br></div><div dir="auto">Where would I find those headers?</div><div dir="auto"><br></div><div dir="auto">Looking at the origin servers apache logs.. it’s sending a 200 response. <br clear="all"><br clear="all"><div dir="auto"><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><br></div><div dir="ltr">Regards,<div><br></div><div>Ben</div></div></div></div></div></div></div><div><br></div><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, 12 Jul 2024 at 18:26, Alex Rousskov <<a href="mailto:rousskov@measurement-factory.com">rousskov@measurement-factory.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;border-left-color:rgb(204,204,204)">On 2024-07-12 13:03, Ben Toms wrote:<br>
<br>
> So the issue seems to be caching content that requires authentication<br>
<br>
The client is getting an error response from Squid. That error is <br>
probably not related to caching decisions. I do not recommend focusing <br>
on caching at this stage of triage. I recommend addressing that error first.<br>
<br>
<br>
> The question here is, can squid cache items that require authentication <br>
> to access?<br>
<br>
Yes, in some cases. To know whether your case qualifies, I asked for the <br>
response headers. That led to the discovery that there are none (from <br>
child Squid point of view). If you really want to investigate the <br>
caching angle in parallel with solving ERR_READ_ERROR/WITH_SERVER, then <br>
try to obtain HTTP response headers that the origin server responds (to <br>
the parent cache) with.<br>
<br>
<br>
HTH,<br>
<br>
Alex.<br>
<br>
<br>
> *From: *Ben Toms <<a href="mailto:ben@macmule.com" target="_blank">ben@macmule.com</a>><br>
> *Date: *Friday, 12 July 2024 at 17:56<br>
> *To: *Alex Rousskov <<a href="mailto:rousskov@measurement-factory.com" target="_blank">rousskov@measurement-factory.com</a>>, <br>
> <a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a> <<a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a>><br>
> *Subject: *Re: [squid-users] TCP_MISS_ABORTED/502<br>
> <br>
> So, with the below config:<br>
> <br>
> https_port 443 accel protocol=HTTPS tls-cert=/usr/local/squid/client.pem <br>
> tls-key=/usr/local/squid/client.key<br>
> <br>
> cache_peer public.server.fqdn parent 443 0 no-query originserver <br>
> no-digest no-netdb-exchange tls login=PASSTHRU name=myAccel <br>
> forceddomain=public.server.fqdn<br>
> <br>
> acl our_sites dstdomain local.server.fqdn<br>
> <br>
> http_access allow our_sites<br>
> <br>
> cache_peer_access myAccel allow our_sites<br>
> <br>
> cache_peer_access myAccel deny all<br>
> <br>
> cache_dir ufs /usr/local/squid/var/cache 100000 16 256<br>
> <br>
> cache_mem 500 MB<br>
> <br>
> maximum_object_size_in_memory 50000 KB<br>
> <br>
> refresh_pattern .               0       20%     4320<br>
> <br>
> debug_options 11,2<br>
> <br>
> I can see the below in /var/log/squid/cache.log<br>
> <br>
> ----------<br>
> <br>
> 2024/07/12 16:49:57.056 kid1| 11,2| http.cc(1263) readReply: conn12 <br>
> local=client.ip:56670 remote=public.ip.of.public.server:443 <br>
> FIRSTUP_PARENT FD 14 flags=1: read failure: (0) No error.<br>
> <br>
> 2024/07/12 16:49:57.056 kid1| 11,2| Stream.cc(273) sendStartOfMessage: <br>
> HTTP Client conn9 local=client.ip:443 remote=local.server.ip:59158 FD 13 <br>
> flags=1<br>
> <br>
> 2024/07/12 16:49:57.056 kid1| 11,2| Stream.cc(274) sendStartOfMessage: <br>
> HTTP Client REPLY:<br>
> <br>
> ---------<br>
> <br>
> HTTP/1.1 502 Bad Gateway<br>
> <br>
> Server: squid/6.6<br>
> <br>
> Mime-Version: 1.0<br>
> <br>
> Date: Fri, 12 Jul 2024 16:49:57 GMT<br>
> <br>
> Content-Type: text/html;charset=utf-8<br>
> <br>
> Content-Length: 3629<br>
> <br>
> X-Squid-Error: ERR_READ_ERROR 0<br>
> <br>
> Vary: Accept-Language<br>
> <br>
> Content-Language: en<br>
> <br>
> Cache-Status: local.server;detail=mismatch<br>
> <br>
> Via: 1.1 local.server (squid/6.6)<br>
> <br>
> Connection: keep-alive<br>
> <br>
> ----------<br>
> <br>
> The apache server still shows a 200 for the request:<br>
> <br>
> [12/Jul/2024:17:49:57 +0100] "GET /path/to/file HTTP/1.1" 200 10465 "-" <br>
> "curl/8.7.1"<br>
> <br>
> And this is when testing via:<br>
> <br>
> curl -D - <a href="https://local.server.fqdn/path/to/file" rel="noreferrer" target="_blank">https://local.server.fqdn/path/to/file</a> <br>
> <<a href="https://local.server.fqdn/path/to/file" rel="noreferrer" target="_blank">https://local.server.fqdn/path/to/file</a>> -H "Authorization: Basic <br>
> base64auth" -o /dev/null<br>
> <br>
> Regards,<br>
> <br>
> Ben.<br>
> <br>
> *From: *Alex Rousskov <<a href="mailto:rousskov@measurement-factory.com" target="_blank">rousskov@measurement-factory.com</a>><br>
> *Date: *Friday, 12 July 2024 at 17:36<br>
> *To: *Ben Toms <<a href="mailto:ben@macmule.com" target="_blank">ben@macmule.com</a>>, <a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a> <br>
> <<a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a>><br>
> *Subject: *Re: [squid-users] TCP_MISS_ABORTED/502<br>
> <br>
> On 2024-07-12 12:14, Ben Toms wrote:<br>
> <br>
>> Which log should those be found?<br>
> <br>
> cache.log (if they are present)<br>
> <br>
> <br>
>> Can’t see “HTTP Server RESPONSE” in the access.log or cache.log.<br>
> <br>
> Sigh. This is one of the reasons I avoid asking folks to study logs<br>
> themselves, even ALL,2 logs...<br>
> <br>
> If that line is not in cache.log, then child Squid probably did not<br>
> receive a response from parent Squid, or could not parse that response.<br>
> A full debugging log should give us more information.<br>
> <br>
> Alex.<br>
> <br>
> <br>
>> *From: *squid-users <<a href="mailto:squid-users-bounces@lists.squid-cache.org" target="_blank">squid-users-bounces@lists.squid-cache.org</a>> on <br>
>> behalf of Alex Rousskov <<a href="mailto:rousskov@measurement-factory.com" target="_blank">rousskov@measurement-factory.com</a>><br>
>> *Date: *Friday, 12 July 2024 at 17:11<br>
>> *To: *<a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a> <<a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a>><br>
>> *Subject: *Re: [squid-users] TCP_MISS_ABORTED/502<br>
>> <br>
>> On 2024-07-12 11:38, Ben Toms wrote:<br>
>>> Think I made the changes Alex requested:<br>
>>> <br>
>>> 12/Jul/2024:15:36:31 +0000.640 local.server.ip TCP_MISS_ABORTED/502 3974 <br>
>>> GET <a href="https://local.server.fqdn/path/to/file" rel="noreferrer" target="_blank">https://local.server.fqdn/path/to/file</a> <br>
> <<a href="https://local.server.fqdn/path/to/file" rel="noreferrer" target="_blank">https://local.server.fqdn/path/to/file</a>><br>
>> <<a href="https://local.server.fqdn/path/to/file" rel="noreferrer" target="_blank">https://local.server.fqdn/path/to/file</a> <br>
> <<a href="https://local.server.fqdn/path/to/file" rel="noreferrer" target="_blank">https://local.server.fqdn/path/to/file</a>>> -<br>
>>> FIRSTUP_PARENT/public.ip.of.public.server text/html <br>
>>> ERR_READ_ERROR/WITH_SERVER<br>
>> <br>
>> Thank you for using Squid v6 for this test.<br>
>> <br>
>> Unfortunately, due to Squid logging bugs, ERR_READ_ERROR/WITH_SERVER<br>
>> does not always mean what it says. For example, parent Squid could have<br>
>> closed the child-parent connection prematurely, but there could be other<br>
>> reasons. A full debugging log should give us more information.<br>
>> <br>
>> <br>
>>> 2024/07/12 14:57:08.678 kid1| 11,2| Stream.cc(274) sendStartOfMessage: <br>
>>> HTTP Client REPLY:<br>
>> <br>
>> This is a child proxy response to the client. We need parent response to<br>
>> the child proxy. Look for "HTTP Server RESPONSE" lines instead.<br>
>> <br>
>> <br>
>> HTH,<br>
>> <br>
>> Alex.<br>
>> <br>
>> <br>
>> <br>
>>> ---------<br>
>>> <br>
>>> HTTP/1.1 502 Bad Gateway<br>
>>> <br>
>>> Server: squid/6.6<br>
>>> <br>
>>> Mime-Version: 1.0<br>
>>> <br>
>>> Date: Fri, 12 Jul 2024 14:57:08 GMT<br>
>>> <br>
>>> Content-Type: text/html;charset=utf-8<br>
>>> <br>
>>> Content-Length: 3629<br>
>>> <br>
>>> X-Squid-Error: ERR_READ_ERROR 0<br>
>>> <br>
>>> Vary: Accept-Language<br>
>>> <br>
>>> Content-Language: en<br>
>>> <br>
>>> Cache-Status: squid.host;detail=mismatch<br>
>>> <br>
>>> Via: 1.1 squid.host (squid/6.6)<br>
>>> <br>
>>> Connection: keep-alive<br>
>>> <br>
>>> ----------<br>
>>> <br>
>>> Regards,<br>
>>> <br>
>>> Ben.<br>
>>> <br>
>>> *From: *squid-users <<a href="mailto:squid-users-bounces@lists.squid-cache.org" target="_blank">squid-users-bounces@lists.squid-cache.org</a>> on <br>
>>> behalf of Amos Jeffries <<a href="mailto:squid3@treenet.co.nz" target="_blank">squid3@treenet.co.nz</a>><br>
>>> *Date: *Friday, 12 July 2024 at 15:22<br>
>>> *To: *<a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a> <<a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a>><br>
>>> *Subject: *Re: [squid-users] TCP_MISS_ABORTED/502<br>
>>> <br>
>>> <br>
>>> On 13/07/24 01:52, Alex Rousskov wrote:<br>
>>>> On 2024-07-12 08:06, Ben Toms wrote:<br>
>>>>> Seems that my issue is similar to - <br>
>>>>> <a href="https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication" rel="noreferrer" target="_blank">https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication</a> <<a href="https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication" rel="noreferrer" target="_blank">https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication</a>> <<a href="https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication" rel="noreferrer" target="_blank">https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication</a> <<a href="https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication" rel="noreferrer" target="_blank">https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication</a>>> <<a href="https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication" rel="noreferrer" target="_blank">https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication</a> <<a href="https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication" rel="noreferrer" target="_blank">https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication</a> <<a href="https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication" rel="noreferrer" target="_blank">https://serverfault.com/questions/1104330/squid-cache-items-behind-basic-authentication</a>>>><br>
>>>> <br>
>>>> You are facing up to two problems:<br>
>>>> <br>
>>>> 1. Some authenticated responses are not cachable by Squid. Please share <br>
>>>> HTTP headers of the response in question.<br>
>>>> <br>
>>> <br>
>>> FYI, those can be obtained by configuring squid.conf with<br>
>>> <br>
>>>     debug_options 11,2<br>
>>> <br>
>>> <br>
>>> Cheers<br>
>>> Amos<br>
>>> <br>
>>> <br>
>>>> 2. TCP_MISS_ABORTED/502 errors may delete a being-cached response. These <br>
>>>> can be bogus errors (essentially Squid logging bugs) or real ones (e.g., <br>
>>>> due to communication bugs, misconfiguration, or compatibility problems). <br>
>>>> I recommend adding %err_code/%err_detail to your logformat and sharing <br>
>>>> the corresponding access.log lines (obfuscated as needed).<br>
>>>> <br>
>>>> Sharing (privately if needed) a pointer to compressed ALL,9 cache.log <br>
>>>> while reproducing the issue using a single transaction may help us <br>
>>>> resolve all the unknowns:<br>
>>>> <br>
>>>> <a href="https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction" rel="noreferrer" target="_blank">https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction</a> <<a href="https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction" rel="noreferrer" target="_blank">https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction</a>> <<a href="https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction" rel="noreferrer" target="_blank">https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction</a> <<a href="https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction" rel="noreferrer" target="_blank">https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction</a>>> <<a href="https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction" rel="noreferrer" target="_blank">https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction</a> <<a href="https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction" rel="noreferrer" target="_blank">https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction</a> <<a href="https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction" rel="noreferrer" target="_blank">https://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction</a>>>><br>
>>>> <br>
>>>> <br>
>>>> HTH,<br>
>>>> <br>
>>>> Alex.<br>
>>>> <br>
>>>> <br>
>>> <br>
>>> <br>
>>> _______________________________________________<br>
>>> squid-users mailing list<br>
>>> <a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a><br>
>>> <a href="https://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">https://lists.squid-cache.org/listinfo/squid-users</a> <br>
> <<a href="https://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">https://lists.squid-cache.org/listinfo/squid-users</a>><br>
>> <<a href="https://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">https://lists.squid-cache.org/listinfo/squid-users</a> <br>
> <<a href="https://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">https://lists.squid-cache.org/listinfo/squid-users</a>>><br>
>>> <<a href="https://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">https://lists.squid-cache.org/listinfo/squid-users</a> <br>
>> <<a href="https://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">https://lists.squid-cache.org/listinfo/squid-users</a> <br>
> <<a href="https://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">https://lists.squid-cache.org/listinfo/squid-users</a>>>><br>
>>> <br>
>>> <br>
>>> _______________________________________________<br>
>>> squid-users mailing list<br>
>>> <a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a><br>
>>> <a href="https://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">https://lists.squid-cache.org/listinfo/squid-users</a> <br>
> <<a href="https://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">https://lists.squid-cache.org/listinfo/squid-users</a>><br>
>> <<a href="https://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">https://lists.squid-cache.org/listinfo/squid-users</a> <br>
> <<a href="https://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">https://lists.squid-cache.org/listinfo/squid-users</a>>><br>
>> <br>
>> _______________________________________________<br>
>> squid-users mailing list<br>
>> <a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a><br>
>> <a href="https://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">https://lists.squid-cache.org/listinfo/squid-users</a> <br>
> <<a href="https://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">https://lists.squid-cache.org/listinfo/squid-users</a>><br>
>> <<a href="https://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">https://lists.squid-cache.org/listinfo/squid-users</a> <br>
> <<a href="https://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">https://lists.squid-cache.org/listinfo/squid-users</a>>><br>
>> <br>
> <br>
<br>
</blockquote></div></div>