[squid-users] Network appliance Sizing for Squid transparent proxy

[Amos Jeffries]

On 27/11/2022 5:37 am, Lucas Vicente Pereira wrote:
> Dear,
>
>
> I have a project and would like to use Squid for WebProxy with Full 
> SSL inspection, external acl to control urls access.
>

Firstly, be aware that TLS when used properly *cannot* be decrypted 
transparently.
So do not expect 100% inspection rates. Even though typical web HTTPS 
can still be inspected, service security levels vary a lot.


> Environment information:
>
> ~2500 users
> 3 x Internet links 1 Gbps each
> Average HTTP requests per minute since start: 65956.1
> ClamAv integration
> Snort Integration
> Iptables REDIRECT for squid
>
> Please, can you please help me with appliance sizing for this environment?
>

Disclaimer:  I do not have any numbers for Squid when ClamAv is added. 
Someone else here may be able to supply or correct my below numbers from 
their actual experience when that tool is used.

AFAIK Snort and iptables are so much more efficient than Squid that they 
are essentially not relevant.


Your 1.1k RPS is well within Squid's capabilities for HTTP (15k-20k RPS 
limit), but the "SSL inspection" will add much overhead so YMMV.

Squid places heavy loads on CPU and I/O systems. The key things to look 
for when you are pushing performance boundaries are (in order of 
impact/gains IMO):

  * as much RAM as you can afford. Within reason, ~128GB is probably 
enough for most Squid.
   - I/O is a major point of performance loss. Network I/O is implicitly 
minimized by Squid defaults.
   - Now that RAM comes by the GB I typically recommend a memory cache 
over disk cache.

  * prefer CPU with higher GHz rating than more cores,
   - the GHz translates directly into faster transaction times + more 
clients, cores translates only to parallel capacity/clients.

  * real/physical cores better than hyper-threading,
    - Squid workers are single-threaded and can push their CPU hard. In 
this case hyper-threading just slows the CPU down.

  * bare-metal better than container;  container better than VM.
   - under load Squid will just keep squeezing the machine until it has 
nothing left to give - then traffic speed takes a nosedive at the worst 
possible time to do so.

The rest is down to configuration and making the whole system as simple 
as possible at every level.


HTH
Amos