[squid-users] Running SMP workers on multi-core systems

[Alex Rousskov]

On 9/30/21 2:49 AM, Praveen Ponakanti wrote:

> I am trying to use the squid cache as an outbound HTTP/S proxy service
> for our production network and would like to scale up the request per
> sec by running it on multi-core systems. 

> I have searched up the archives and config docs, but couldn't find
> anything more recent than the below on setting up SMP workers. 
> https://wiki.squid-cache.org/ConfigExamples/SmpCarpCluster
> https://wiki.squid-cache.org/ConfigExamples/MultiCpuSystem

The two old exotic configurations above are not what you should be
starting with when thinking about SMP support in modern Squids. The best
starting points I know about are

* https://wiki.squid-cache.org/Features/SmpScale
* http://www.squid-cache.org/Doc/config/workers/
* http://www.squid-cache.org/Doc/config/cpu_affinity_map/
* http(s)_port worker-queues option

That Feature wiki page does not get many updates these days; if there is
a conflict between information sources, the directive documentation in
your squid.conf.documented may have more recent information.


> * Can someone please let me know if the config examples in those docs
> still apply to the latest version 5.1.

I bet those use cases themselves do not apply well to your situation.


> * Do we still need the combination of squid, frontend, backend.conf
> files, or is it enabled simply with the "workers" line in the main
> config file? 

The workers directive does not require multiple Squid instances (with
multiple configuration files, etc.).


> * When I attempt to run the squid server with "workers 4" in the
> squid.conf, I get the following error (one for each kid) and the main
> process does not bind to the "http_port 3128". I see several UDP ports
> opened up for each kid process.

> 2021/09/29 00:44:10 kid5| commBind Cannot bind socket FD 11 to [::]: (2) No such file or directory

Unfortunately, I cannot tell exactly what went wrong based on that
low-level message alone, but it could be a variant of [1]. If you cannot
figure it out after checking [1] suggestions, consider sharing "squid
-X" startup cache.log for analysis. There is

[1]
https://wiki.squid-cache.org/Features/SmpScale#Cannot_bind_socket_FD_NN_to_.5B::.5D:_.2813.29_Permission_denied

The UDP ports you are seeing are probably for the internal DNS resolver
which is not SMP-aware.


> * Does anyone have recommendations on the maximum number of workers to
> use on a 64 core host (assuming no other CPU intensive apps are running
> on the same host). Caching is not a must for our initial deployment, so
> we are fine disabling caching.

28-30. See the following wiki section for the corresponding rules of
thumb:
https://wiki.squid-cache.org/Features/SmpScale#How_to_configure_SMP_Squid_for_top_performance.3F


HTH,

Alex.