[squid-users] ICAP Persistent Connections vs Retries (with code review)
squid3 at treenet.co.nz
Fri May 12 01:30:39 UTC 2017
On 12/05/17 12:49, Juan Ramírez wrote:
> Mi name is Juan, I am a Software Engineer from Uruguay. I think this
> message is more suited to the squid-dev mailing list due to the
> developer-oriented nature of the message but, given that the
> development list is for people who actually contributes code to Squid,
> I chose to post here.
Membership is for those people, the list itself acts as a focal point
for anyone such as yourself with this query to contact them without
having to know particular peoples contacts.
As to your issue;
Requests which are not retriable are not able to be re-sent if it turns
out they got even partially delivered on a pre-opened persistent
connection which happened to be in the process of closing by the other
endpoint unknown to Squid. For example if the timing of the write to
deliver the ICAP RESPMOD headers overlapped with the TCP FIN/RST packets
coming from a router along the network path, or the ICAP service itself.
As such those requests need a new TCP connection to be opened to
guarantee the absence of immediate closure. When they complete with
their transaction it gets added to the pool for other traffic to use if
The behaviour which you describe was designed to prevent the ICAP pool
from absorbing all available TCP resources through these constant brand
new TCP connections being needed then added to the pool. Which would
leave the proxy unable to receive HTTP traffic as there would be no TCP
sockets/ports available for non-ICAP connections.
If your service only (or mostly) has non-retriable transactions occuring
it can be more efficient to disable persistence o speed up the turn
around on TCP socket closures. Persistence is useful for when you have
many retriable transactions occuring which can use pooled connections
and be re-tried if something barfs at the TCP level.
As far as I know RESPMOD transactions should all be retriable unless the
body payload length is unknown at the HTTP level (lack of Content-Length
They can also become non-retriable when the response from the ICAP
server has been received by Squid and already started delivery to the
HTTP client. That is the noteCommRead call - to prevent the connection
being pooled if the connection suddenly closes before it completes. This
should not have any effect on whether a pooled connection was used to
start with, but will affect whether a pooled connection gets used on the
retry attempt to prevent the same issue/delay repeating indefinitely to
the same client.
(PS. Alex knows a lot more about the details of this design than me, so
if he says anything contradicting the above go with his).
More information about the squid-users