[squid-users] Understanding maximum outgoing HTTP CONNECT requests?

[Amos Jeffries]

On 18/02/2023 12:14 am, divan.whelk.0u wrote:
> Hi there!
>
> I’m trying to understand what would the “theoretical” maximum amount of outgoing connections with squid setup as a HTTP CONNECT forward proxy would be (hardware permitting)?

As you likely know, each TCP/IP connection uses a 4-tuple identifier 
{src-IP, src-port, dst-IP, dst-port}.

So at face value there is a protocol imposed cap of (2^128 * 2^16 * 
2^128 * 2^16) = 2^288 connections.

Being theoretical we have:
     * ignored reserved IP ranges,
     * ignored OS-specific ephemeral port reservations,
     * assumed IPv6 availability, and
     * assumed no access restrictions in Squid, network routing, nor 
firewall.

The factors to consider are:

  - Squid machine can be assigned multiple IP's.
     Thus src-IP on outbound and dst-IP on inbound are that N.

  - Squid can be configured to receive on up to 64 ports.
    Thus dst-port on inbound is 2^6.

  - DNS can provide any number of IPs for any given server name.
     Thus outbound dst-IP can be any 2^128 value.

  - modern websites use use Alt-Svc to spread across ports.
     Thus outbound dst-port can be any 2^16 value.

So for theoretical limit the math is:

  inbound =    2^128 * 2^16 * N * 2^16  = N * 2^160

  outbound =  N * 2^6 * 2^128 * 2^16 = N * 2^150

Inbound and outbound are normally independent, but CONNECT is a special 
case where they are pinned 1:1.

Thus total theoretical limit of simultaneous connections Squid can be 
juggling is  N * 2^151.

Reality can be significantly different for any given installation, but 
is imposed by configuration choices and thus can be altered as needed.


>  From the [squid-users] About bottlenecks (Max number of connections, etc.) thread, I saw mention of the following:
>
>> * The limit on number of connections any Squid can have attached is only limited by your configured FD limits and available server RAM. Squid uses ~64 KB per network socket for traffic state - which equates to around 2 GB of RAM just for I/O buffers at 20,000 concurrent client connections.
> I assume the same would not apply on outgoing connections, and that there would be a limit of 65,536 connections to a single IP, port pair? For example, if we had 1 million users making requests via HTTP CONNECT, only 65K of them would be able to access the same website at any one time?

IIRC that quoted thread was discussing a Squid with more normal 
multiple-destination case hitting FD limits.  The 64K port limitation 
you refer to is a special case contingent on the "single destination 
with single IP:port" criteria - which itself is rarely true for a 
popular website. It assumes configuration restriction imposing that 
criteria somehow.


Cheers
Amos