[squid-users] ICAP Persistent Connections vs Retries (with code review)

[Alex Rousskov]

On 05/12/2017 01:17 PM, Juan Ramírez wrote:

> I still don't understand whether it is possible to reuse ICAP
> connections for cases other than retries.

You are implying that idle persistent connections are used for retries.
They are not (or, at least, should not be). Idle persistent connections
are used for new requests that can be retried.

I am deleting most of my carefully written answers and skipping most of
your email until the place where you discovered problems without your
earlier statements/questions :-(.


> * Why does the size of the payload affect the way connections are reused

Because to resend the request, Squid has to have the payload to send.
Thus, to resend a 10GB request, Squid has to buffer a 10GB request.
Squid does not want to do that for, hopefully, obvious reasons.


> * Does Squid need a finer control on this matter in order to ensure
> connection reusability is not affected by the size of the payload?

No. Connection reusability is affected by the size of the payload. It is
the unavoidable consequence of combining finite memory with potentially
infinite response sizes.


> I understand much more now than when I started writing this message.

FWIW, you would save some of us a lot of time if you do not send
portions that you later (but in the same email!) discovered to be
incorrect or irrelevant. This mailing list is not a blog; somebody may
actually be reading and responding to what you write.


> + I am not sure if it needs finer control, but the fact that is closing
> connections in advance seems dangerous.

There is no danger in a client closing an idle connection.


> + It also seems dangerous to me this protection mechanism given the fact
> that opening/closing TCP connections everytime can become a bottleneck.

There is some performance overhead, but not danger. High quality
optimizations are welcomed.


> I disabled the call to `theIdleConns->closeN(1);` 
> Do you think there would be any issues with this change?

Yes, I do:

* In some environments, your patched Squid is likely to open too many
connections, violating various OS limits and/or admin expectations.

* In some environments, your patched Squid is likely to stop servicing
requests after hitting one of the open connection limits, unable to open
a new connection for an (unretriable) transaction, with a connection
pool full of idle connections.

The code you do not like was added because Squid was experiencing such
problems in some real-world environments (both HTTP and ICAP). It _can_
be improved, and quality patches are welcomed, but it should not be
disabled (in the official version).


HTH,

Alex.