[squid-users] persistent connections not being utilized with Chrome

[Alex Rousskov]

On 01/12/2018 08:23 PM, Brian J. Murrell wrote:

> I mean, will not actually result in a persistent connection -- a socket
> that is reused for multiple HTTP transactions. 

It is best to think of HTTP persistency as applying to an HTTP
connection rather than a TCP connection/socket. Without a proxy, your
definition works well. With a proxy, there are several different use
cases. The one you seem to be interested in is encrypted HTTP connection
through an explicit proxy. In that case, there are two HTTP connections
in play:

  1. An HTTP connection from the client to the origin server.
  2. An HTTP connection from the client to the proxy.

Both HTTP connections use the same TCP client connection/socket (there
is also another TCP server connection socket that HTTP connection #1
also uses).

The first HTTP connection can carry lots of transactions. It can easily
be and often is "persistent".

The second HTTP connection typically carries just one HTTP transaction
(a CONNECT request followed by a 200 OK response establishing a TCP
tunnel through the proxy), but that is OK -- it does not hurt
performance because the useful transactions utilize HTTP connection #1
(which lives inside the tunnel established by the HTTP connection #2).


> I suppose for CONNECT
> it would mean either multiple CONNECTs within a single socket or one
> CONNECT with multiple "GET/POST" type transactions within it.

Authentication and other corner cases aside, there can be at most one
CONNECT request per HTTP connection, but you do not care (much) about
_that_ HTTP connection. The other HTTP connection (the one inside the
CONNECT tunnel) can carry lots of requests, and those transactions
actually deliver "pages" to your browser.


> So, for the purposes of the WWW's current move towards all-https
> websites, persistent connections (perhaps only with proxy servers?) are
> becoming useless?

No, the optimization is still there as far as client-origin traffic is
concerned.


> The problem I am trying to solve here is that opening Chrome with, say,
> a few hundred tabs open, seems to take about an hour for it to finally
> fetch all of the pages while most sit spinning on "waiting for an
> available socket" or "proxy tunnel", etc., for a long time, which is
> probably due to Chrome's limit on the number of concurrent sockets it
> will open to a single destination, including proxy servers.  

Yes, probably something like that is happening.


> I was hoping persistent connections would reduce the socket
> setup/teardown overhead of all of that.

Perhaps they do? How many requests does Chrome send inside a CONNECT
tunnel through Squid, on average? If you bump CONNECT tunnels using
SslBump, then you can use Squid to measure persistency. If you do not
bump, then you should still be able to use Chrome developer tools to
measure persistency.

Origin server response delays rather than TCP handshakes may be your
primary bottleneck because Chrome probably does not pipeline and,
without pipelining, there can be at most one concurrent request per HTTP
connection. To improve throughput in your environment (without raising
the number of TCP connections that Chrome is allowed to open), you would
need to wait for HTTP/2 support. In HTTP/2, a single HTTP connection can
carry lots of concurrent transactions.


HTH,

Alex.