<div dir="ltr"><div>>As with any timeout, it is impossible to say in general which side of</div>>the connection is at fault. This case has at least three sides: It could<br>>be the HTTP agent, Squid, and/or the ICAP service. Did one of them stall<br>>the transaction? Or was the ICAP service just too impatient? See option<br>>#4 below.<div>I've tried to track this down. There are some persistent sockets used by SaaS apps for APIs (otservice api from google sites) and sometimes the HTTP response takes a long time to trickle in. I have seen upto 25 seconds for the response body to trickle in after the response header. I don't know yet if this is due to network delays but given that it happens only for this particular uri I'm theorizing that this is how it works. I can whitelist the one i am aware of that is causing this issue but again the concern is what about others that might throw this exception. The timeout at 10 seconds is somewhat aggressive so moving that up should help but some code changes in either squid or icap as suggested look necessary.</div><div><br><div>>Please note that ICAP is a protocol, not a product/software name. It<br></div>>probably does not matter what ICAP service you are using though.</div><div><br><div>Correct. I was referring to the c-icap implementation.<br><br><div class="gmail_quote"><div dir="ltr">On Wed, Oct 31, 2018 at 5:00 AM <<a href="mailto:squid-users-request@lists.squid-cache.org" target="_blank">squid-users-request@lists.squid-cache.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Send squid-users mailing list submissions to<br>
<a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a><br>
<br>
To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:squid-users-request@lists.squid-cache.org" target="_blank">squid-users-request@lists.squid-cache.org</a><br>
<br>
You can reach the person managing the list at<br>
<a href="mailto:squid-users-owner@lists.squid-cache.org" target="_blank">squid-users-owner@lists.squid-cache.org</a><br>
<br>
When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of squid-users digest..."<br>
<br>
<br>
Today's Topics:<br>
<br>
1. <br>
On 10/30/18 6:45 PM, Arunabha Saha wrote:<br>
<br>
> Squid 3.5.25 does not seem to recognise the 408 request timeout error<br>
> code from ICAP.<br>
<br>
Squid effectively recognizes ICAP 408 response as an ICAP transaction<br>
error response and blames the ICAP service for that error. That<br>
(minimal) support can be improved, of course. See options #1 and #3 below.<br>
<br>
<br></blockquote><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Needless to say, treating all ICAP service timeouts as if nothing bad<br>
happened would break some existing Squid deployments (while possibly<br>
fixing yours). A proper general solution (option #3 below) would most<br>
likely require making Squid behavior configurable.<br>
<br>
<br>
> The more troublesome issue for me is the exception it generates and then<br>
> declares ICAP down after a certain number of such exceptions. <br>
> <br>
> I don't want to disable the failure limit entirely given that we can<br>
> often have genuine failures that squid needs to detect. <br>
> <br>
> What i'd like to see is squid should not throw an exception in this<br>
> case.<br>
<br>
The "exception" is a minor low-level/technical detail. What you really<br>
want to see is Squid blaming itself (rather than the ICAP service) for<br>
the problem. Squid indeed lacks that kind of functionality, but it can<br>
be added if really needed. See options #1 and #3 below.<br>
<br>
<br>
> The timeout is somewhat aggressive but works with an earlier<br>
> version of ICAP (0.1.x). The one i'm testing is 0.5.3.<br>
<br>
Please note that ICAP is a protocol, not a product/software name. It<br>
probably does not matter what ICAP service you are using though.<br>
<br>
<br>
> Any suggestions?<br>
<br>
I can suggest a few options, in no particular order:<br>
<br>
1. Modify your Squid to treat 408 differently.<br>
2. Modify your ICAP service to stop sending ICAP 408 responses to Squid.<br>
3. Add proper ICAP timeout support feature to Squid.<br>
4. Investigate why your ICAP service times out. If you are lucky, you<br>
may be able to fix or work around the problem by adjusting Squid<br>
and/or your ICAP service configuration.<br>
<br>
For option #1, Adaptation::Icap::ModXact::parseIcapHead() may be a good<br>
starting point.<br>
<br>
For options #1 and #3, see also<br>
<a href="https://wiki.squid-cache.org/SquidFaq/AboutSquid#How_to_add_a_new_Squid_feature.2C_enhance.2C_of_fix_something.3F" rel="noreferrer" target="_blank">https://wiki.squid-cache.org/SquidFaq/AboutSquid#How_to_add_a_new_Squid_feature.2C_enhance.2C_of_fix_something.3F</a><br>
<br>
In most cases, option #4 is the best first step but YMMV.<br>
<br>
<br>
HTH,<br>
<br>
Alex.<br>
<br>
<br>
------------------------------<br>
<br>
Message: 2<br>
Date: Tue, 30 Oct 2018 23:59:18 -0500 (CDT)<br>
From: Sid <<a href="mailto:SIDDH05@gmail.com" target="_blank">SIDDH05@gmail.com</a>><br>
To: <a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a><br>
Subject: Re: [squid-users] Squid 4.3: SSL Bump fails to send client<br>
certificate<br>
Message-ID: <<a href="mailto:1540961958277-0.post@n4.nabble.com" target="_blank">1540961958277-0.post@n4.nabble.com</a>><br>
Content-Type: text/plain; charset=us-ascii<br>
<br>
Thank you Alex for the reply.<br>
<br>
Alex: 1. Servers never send SNI. Clients usually send SNI. Squid should<br>
forward SNI it received from the client to the server, provided the client<br>
actually sent SNI. Did your client send SNI? <br>
<br>
Sid: I can see in Client Hello IP Address being sent by Client; so there is<br>
no SNI from client itself.<br>
<br>
Alex: 2. Bugs notwithstanding, the implied order of events is not what<br>
actually happens: Squid, as configured, does _not_ forward anything from the<br>
server certificate to the client. Squid, as configured, generates a<br>
certificate based on client-supplied information (not server-supplied<br>
information). After sending that generated certificate to the client, Squid<br>
establishes a TLS connection with the server. <br>
<br>
Sid: Thank you for explanation.<br>
<br>
Alex: For an accurate picture, in addition to Squid-server and server-Squid<br>
traffic, look at what Squid has received from the client and what Squid sent<br>
to the client, all in actual order.<br>
<br>
Sid: I took wireshark on Squid server (centOS 7); I took 2 wiresharks<br>
between Client & Squid and then between Squid & Server. I can see client<br>
being sent fake cert generated by Squid & client responds with "Client key<br>
Exchange", "Change cipher spec", "Encrypted Handshake Message". But I can't<br>
see actual client certificate sent to Squid. Is there a way to decypt in<br>
Wireshark. In Wireshark between Squid & Server I can see Squid responding<br>
with "61 Alert (Level: Fatal, Description: Internal Error)".<br>
<br>
Alex: Is your Squid configured to trust those internal CAs? If not, Squid<br>
would not be able to validate the server certificate. <br>
<br>
Sid: I have added those chained certificates as following in squid.conf<br>
tls_outgoing_options cafile=/usr/local/squid/etc/UCAppsCA.pem<br>
sslproxy_foreign_intermediate_certs /usr/local/squid/etc/UCAppsCA.pem<br>
<br>
<br>
<br>
<br>
--<br>
Sent from: <a href="http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html" rel="noreferrer" target="_blank">http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html</a><br>
<br>
<br>
------------------------------<br>
<br>
Message: 3<br>
Date: Wed, 31 Oct 2018 05:27:20 -0500 (CDT)<br>
From: uppsalanet <<a href="mailto:fredrik@pipemore.se" target="_blank">fredrik@pipemore.se</a>><br>
To: <a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a><br>
Subject: Re: [squid-users] redirect based on url (302)<br>
Message-ID: <<a href="mailto:1540981640859-0.post@n4.nabble.com" target="_blank">1540981640859-0.post@n4.nabble.com</a>><br>
Content-Type: text/plain; charset=us-ascii<br>
<br>
Hi Amos,<br>
Is there a git that I can use to push stuff up?<br>
<br>
I think you need to split the string in an other way, look into this<br>
example:<br>
#!/usr/bin/perl<br>
use strict;<br>
use warnings;<br>
<br>
$|=1;<br>
while (<>) {<br>
my $string = $_;<br>
print "Received '\$_' = ".$_."\n"; <br>
<br>
$string =~ m/^(\d+)\s(.*)$/;<br>
print "After regexp '\$string' = ".$string."\n";<br>
print "After regexp '\$1' = ".$1."\n"; <br>
print "After regexp '\$2' = ".$2."\n"; <br>
<br>
### Original split from sorce ###<br>
### This doesn't split anything looks like elements of an array?<br>
#my ($cid, $uid) = ($1, $2);<br>
<br>
### Split the string ###<br>
### Those two split based on one or more spaces<br>
#my ($cid, $uid) = split(/\s+/ ,$_);<br>
my ($cid, $uid) = split;<br>
$cid =~ s/%(..)/pack("H*", $1)/ge;<br>
$uid =~ s/%(..)/pack("H*", $1)/ge;<br>
print "After split \$cid = ".$cid."\n";<br>
print "After split \$uid = ".$uid."\n";<br>
}<br>
<br>
Output from above with intake value '*130.238.000.00 <a href="http://muse.jhu.edu" rel="noreferrer" target="_blank">muse.jhu.edu</a> -*':<br>
Received '$_' = 130.238.000.00 <a href="http://muse.jhu.edu" rel="noreferrer" target="_blank">muse.jhu.edu</a> -<br>
After regexp '$string' = 130.238.000.00 <a href="http://muse.jhu.edu" rel="noreferrer" target="_blank">muse.jhu.edu</a> -<br>
/Use of uninitialized value $1 in concatenation (.) or string at<br>
./<a href="http://sed_test_reg.pl" rel="noreferrer" target="_blank">sed_test_reg.pl</a> line 13, <> line 1.<br>
After regexp '$1' = <br>
Use of uninitialized value $2 in concatenation (.) or string at<br>
./<a href="http://sed_test_reg.pl" rel="noreferrer" target="_blank">sed_test_reg.pl</a> line 14, <> line 1.<br>
After regexp '$2' = /<br>
*After split $cid = 130.238.000.00<br>
After split $uid = <a href="http://muse.jhu.edu" rel="noreferrer" target="_blank">muse.jhu.edu</a>*<br>
<br>
Cheers<br>
Fredrik<br>
<br>
<br>
<br>
--<br>
Sent from: <a href="http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html" rel="noreferrer" target="_blank">http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html</a><br>
<br>
<br>
------------------------------<br>
<br>
Subject: Digest Footer<br>
<br>
_______________________________________________<br>
squid-users mailing list<br>
<a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a><br>
<a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a><br>
<br>
<br>
------------------------------<br>
<br>
End of squid-users Digest, Vol 50, Issue 75<br>
*******************************************<br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="m_2831788150584372882gmail_signature" data-smartmail="gmail_signature">regards,<br>Arun</div></div></div></div>