[squid-users] The right way how to increase max_filedescriptors on Linux

[Amos Jeffries]

On 22/05/18 00:08, kAja Ziegler wrote:
> Hi,
> 
>   I want to ask, if it is really needed to use ulimit or
> /etc/security/limits.conf to increase max_filedescriptors value? From my
> testing, it seems not.

Sometimes yes, sometimes no. It depends on what the systems normal
settings are and whether the Squid binary was built with full or partial
rlimit() support.

> 
> 
> *= my environment:*
> 
> CentOS 6.9
> Squid 3.1.23 / 3.4.14
> 
> *- default ulimits for root and other users:*
> 
> [root at ...]# ulimit -Sa | grep -- '-n'
> open files                      (-n) 1024
> [root at ...]# ulimit -Ha | grep -- '-n'
> open files                      (-n) 4096
> 
> *- default ulimits for squid user:*
> 
> [root at ...]# sudo -u squid /bin/bash
> bash-4.1$ id
> uid=23(squid) gid=23(squid) groups=23(squid),...
> bash-4.1$ ulimit -Sa | grep -- '-n'
> open files                      (-n) 1024
> bash-4.1$ ulimit -Ha | grep -- '-n'
> open files                      (-n) 4096
> 
> *- processes:*
> 
> [root at ...]# ps aux | grep squid
> root      7194  0.0  0.1  73524  3492 ?        Ss   May17   0:00 squid
> -f /etc/squid/squid.conf
> squid     7197  0.2 10.9 276080 210156 ?       S    May17   4:53 (squid)
> -f /etc/squid/squid.conf
> squid     7198  0.0  0.0  20080  1084 ?        S    May17   0:00 (unlinkd)
> 
> *- error and warning messages from cache.log:*
> 
> client_side.cc(3070) okToAccept: WARNING! Your cache is running out of
> filedescriptors
> 
> comm_open: socket failure: (24) Too many open files
> 
> IpIntercept.cc(137) NetfilterInterception:  NF
> getsockopt(SO_ORIGINAL_DST) failed on FD 68: (2) No such file or
> directory ... (many with different FD)
> 

These should not be related to FD numbers running out. As you can see FD
68 was already allocated to this TCP connection and the socket accept()'ed.

NAT errors are usually caused by explicit-proxy traffic arriving at a
NAT interception port. Such traffic is prohibited.
or by NAT table overflowing under extreme traffic loads. Either way
current Squid versions will terminate that connection immediately since
it cannot identify where the packets were supposed to be going.

> 
> I found many How-tos like these -
> https://access.redhat.com/solutions/63027 and
> https://www.cyberciti.biz/faq/squid-proxy-server-running-out-filedescriptors/.
> Both how-tos mention editing the file /etc/security/limits.conf and
> adding the line "* - nofile 4096" to increase the nofile limit for all
> users except root - I don't like this. According to my test, see below, 
> this is not necessary, but I want to be sure, so I'm writing here.

Note that neither of those are the official Squid FAQ.

The official recommendation is to use those data sources to *check* what
the system limits are.

The official Best Practice varies depending on ones needs. Packagers
distributing Squid are advised to set reasonable limits in the init
script starting Squid. End users to use the configuration file best
suited to their need (MAY be limits.conf, but usually squid.conf).


> 
> *a) Squid default configuration (max_filedesc 0) and default nofile
> limit (1024/4096):*
> 

Do not set the limit to "0". That actually means *no* filedescriptors
for the newer Squid versions.

Remove the directive entirely from your squid.conf for the default
behaviour.

Also "max_filedescriptors" is teh directive name. "max_filedesc" was
only for the experimental RHEL patch many decades ago.

> 
> *c) Squid configuration with max_filedesc 8192 and default nofile limit
> (1024/4096):*
> 
> [root at ...]# ps aux | grep squid
> root     18734  0.0  0.1  73524  3492 ?        Ss   14:00   0:00 squid
> -f /etc/squid/squid.conf
> squid    18737  0.3  0.6  80244 11860 ?        S    14:00   0:00 (squid)
> -f /etc/squid/squid.conf
> squid    18740  0.0  0.0  20080  1088 ?        S    14:00   0:00 (unlinkd)
> 
> [root at ...]# grep -E "Limit|Max open files" /proc/18734/limits
> Limit                     Soft Limit           Hard Limit           Units
> Max open files            1024                 4096                 files
> 
> [root at ...]# grep -E "Limit|Max open files" /proc/18737/limits
> Limit                     Soft Limit           Hard Limit           Units
> Max open files            *8192*                 *8192*                
> files
> 
> [root at ...]# grep -E "Limit|Max open files" /proc/18740/limits
> Limit                     Soft Limit           Hard Limit           Units
> Max open files            *8192*                 *8192*                
> files
> 
> - both soft and hard nofile limits were increased for processes running
> under squid user
> 
> 
> I think, that the limits could be increased in tests b) and c) because
> the master process runs under the root user. Am I right or not?

AFAIK, Hard limit can be changed by root (or ulimit tool itself would
not work). Soft limit can be changed by any user to any value below hard
limit.

What you see in (c) is the master process changing the hard limit for
its spawned child processes so that they can use the value in squid.conf
without errors.


> Or need I to increase the limits for the master proccess too?

Not if squid is correctly setting the limits for you. Doing that
automatically is one of the reasons the master exists separately from
the workers. The init script use of ulimit is a workaround for builds
where rlimit() support is lacking or broken.

Amos