<div dir="ltr">On Mon, May 21, 2018 at 3:29 PM, Amos Jeffries <span dir="ltr"><<a href="mailto:squid3@treenet.co.nz" target="_blank">squid3@treenet.co.nz</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On 22/05/18 00:08, kAja Ziegler wrote:<br>
> Hi,<br>
> <br>
> I want to ask, if it is really needed to use ulimit or<br>
> /etc/security/limits.conf to increase max_filedescriptors value? From my<br>
> testing, it seems not.<br>
<br>
Sometimes yes, sometimes no. It depends on what the systems normal<br>
settings are and whether the Squid binary was built with full or partial<br>
rlimit() support.<br>
<br>
> <br>
> <br>
> *= my environment:*<br>
> <br>
> CentOS 6.9<br>
> Squid 3.1.23 / 3.4.14<br>
> <br>
> *- default ulimits for root and other users:*<br>
> <br>
> [root@...]# ulimit -Sa | grep -- '-n'<br>
> open files (-n) 1024<br>
> [root@...]# ulimit -Ha | grep -- '-n'<br>
> open files (-n) 4096<br>
> <br>
> *- default ulimits for squid user:*<br>
> <br>
> [root@...]# sudo -u squid /bin/bash<br>
> bash-4.1$ id<br>
> uid=23(squid) gid=23(squid) groups=23(squid),...<br>
> bash-4.1$ ulimit -Sa | grep -- '-n'<br>
> open files (-n) 1024<br>
> bash-4.1$ ulimit -Ha | grep -- '-n'<br>
> open files (-n) 4096<br>
> <br>
> *- processes:*<br>
> <br>
> [root@...]# ps aux | grep squid<br>
> root 7194 0.0 0.1 73524 3492 ? Ss May17 0:00 squid<br>
> -f /etc/squid/squid.conf<br>
> squid 7197 0.2 10.9 276080 210156 ? S May17 4:53 (squid)<br>
> -f /etc/squid/squid.conf<br>
> squid 7198 0.0 0.0 20080 1084 ? S May17 0:00 (unlinkd)<br>
> <br>
> *- error and warning messages from cache.log:*<br>
> <br>
> client_side.cc(3070) okToAccept: WARNING! Your cache is running out of<br>
> filedescriptors<br>
> <br>
> comm_open: socket failure: (24) Too many open files<br>
> <br>
> IpIntercept.cc(137) NetfilterInterception: NF<br>
> getsockopt(SO_ORIGINAL_DST) failed on FD 68: (2) No such file or<br>
> directory ... (many with different FD)<br>
> <br>
<br>
These should not be related to FD numbers running out. As you can see FD<br>
68 was already allocated to this TCP connection and the socket accept()'ed.<br>
<br>
NAT errors are usually caused by explicit-proxy traffic arriving at a<br>
NAT interception port. Such traffic is prohibited.<br>
or by NAT table overflowing under extreme traffic loads. Either way<br>
current Squid versions will terminate that connection immediately since<br>
it cannot identify where the packets were supposed to be going.<br>
<br>
> <br>
> I found many How-tos like these -<br>
> <a href="https://access.redhat.com/solutions/63027" rel="noreferrer" target="_blank">https://access.redhat.com/<wbr>solutions/63027</a> and<br>
> <a href="https://www.cyberciti.biz/faq/squid-proxy-server-running-out-filedescriptors/" rel="noreferrer" target="_blank">https://www.cyberciti.biz/faq/<wbr>squid-proxy-server-running-<wbr>out-filedescriptors/</a>.<br>
> Both how-tos mention editing the file /etc/security/limits.conf and<br>
> adding the line "* - nofile 4096" to increase the nofile limit for all<br>
> users except root - I don't like this. According to my test, see below, <br>
> this is not necessary, but I want to be sure, so I'm writing here.<br>
<br>
Note that neither of those are the official Squid FAQ.<br>
<br>
The official recommendation is to use those data sources to *check* what<br>
the system limits are.<br>
<br>
The official Best Practice varies depending on ones needs. Packagers<br>
distributing Squid are advised to set reasonable limits in the init<br>
script starting Squid. End users to use the configuration file best<br>
suited to their need (MAY be limits.conf, but usually squid.conf).<br>
<br>
<br>
> <br>
> *a) Squid default configuration (max_filedesc 0) and default nofile<br>
> limit (1024/4096):*<br>
> <br>
<br>
Do not set the limit to "0". That actually means *no* filedescriptors<br>
for the newer Squid versions.<br>
<br>
Remove the directive entirely from your squid.conf for the default<br>
behaviour.<br>
<br>
Also "max_filedescriptors" is teh directive name. "max_filedesc" was<br>
only for the experimental RHEL patch many decades ago.<br>
<br>
> <br>
> *c) Squid configuration with max_filedesc 8192 and default nofile limit<br>
> (1024/4096):*<br>
> <br>
> [root@...]# ps aux | grep squid<br>
> root 18734 0.0 0.1 73524 3492 ? Ss 14:00 0:00 squid<br>
> -f /etc/squid/squid.conf<br>
> squid 18737 0.3 0.6 80244 11860 ? S 14:00 0:00 (squid)<br>
> -f /etc/squid/squid.conf<br>
> squid 18740 0.0 0.0 20080 1088 ? S 14:00 0:00 (unlinkd)<br>
> <br>
> [root@...]# grep -E "Limit|Max open files" /proc/18734/limits<br>
> Limit Soft Limit Hard Limit Units<br>
> Max open files 1024 4096 files<br>
> <br>
> [root@...]# grep -E "Limit|Max open files" /proc/18737/limits<br>
> Limit Soft Limit Hard Limit Units<br>
> Max open files *8192* *8192* <br>
> files<br>
> <br>
> [root@...]# grep -E "Limit|Max open files" /proc/18740/limits<br>
> Limit Soft Limit Hard Limit Units<br>
> Max open files *8192* <wbr> *8192* <wbr> <br>
> files<br>
> <br>
> - both soft and hard nofile limits were increased for processes running<br>
> under squid user<br>
> <br>
> <br>
> I think, that the limits could be increased in tests b) and c) because<br>
> the master process runs under the root user. Am I right or not?<br>
<br>
AFAIK, Hard limit can be changed by root (or ulimit tool itself would<br>
not work). Soft limit can be changed by any user to any value below hard<br>
limit.<br>
<br>
What you see in (c) is the master process changing the hard limit for<br>
its spawned child processes so that they can use the value in squid.conf<br>
without errors.<br>
<br>
<br>
> Or need I to increase the limits for the master proccess too?<br>
<br>
Not if squid is correctly setting the limits for you. Doing that<br>
automatically is one of the reasons the master exists separately from<br>
the workers. The init script use of ulimit is a workaround for builds<br>
where rlimit() support is lacking or broken.<br>
<br>
Amos<br>
______________________________<wbr>_________________<br>
squid-users mailing list<br>
<a href="mailto:squid-users@lists.squid-cache.org">squid-users@lists.squid-cache.<wbr>org</a><br>
<a href="http://lists.squid-cache.org/listinfo/squid-users" rel="noreferrer" target="_blank">http://lists.squid-cache.org/<wbr>listinfo/squid-users</a><br>
</blockquote></div></div><div class="gmail_extra"><br></div><div class="gmail_extra"><br></div><div class="gmail_extra"><span id="gmail-result_box" class="gmail-" lang="en"><span class="gmail-">Thank you Amos for your <span id="gmail-result_box" class="gmail-short_text" lang="en"><span class="gmail-">clarification</span></span>/explanation and confirmation of my presumptions.</span></span><br></div><div class="gmail_extra"><br></div><div class="gmail_extra"><span id="gmail-result_box" class="gmail-short_text" lang="en"><span class="gmail-alt-edited">About the NAT errors I'm going to write new email.</span></span></div><div class="gmail_extra"><span id="gmail-result_box" class="gmail-short_text" lang="en"><span class="gmail-alt-edited"><br></span></span></div><div class="gmail_extra"><span id="gmail-result_box" class="gmail-short_text" lang="en"><span class="gmail-alt-edited"></span></span><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Do not set the limit to "0". That actually means *no* filedescriptors for the newer Squid versions.<br><br>Remove the directive entirely from your squid.conf for the default behaviour.</blockquote></div><div class="gmail_extra"><br></div><div class="gmail_extra">Thank you for the warning, I came up with the documentation for Squid 3.1.<br></div><div class="gmail_extra"><br></div><div class="gmail_extra"><br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Also "max_filedescriptors" is teh directive name. "max_filedesc" was only for the experimental RHEL patch many decades ago.</blockquote></div><div class="gmail_extra"><br></div><div class="gmail_extra">Yep, I made copy & paste error from the old RHEL 5.x page - <a href="https://access.redhat.com/solutions/63027" rel="noreferrer" target="_blank">https://access.redhat.com/<wbr>solutions/63027</a>. <div class="gmail_extra">I use <span id="gmail-result_box" class="gmail-short_text" lang="en"><span class="gmail-">"max_filedescriptors"</span></span> <span id="gmail-result_box" class="gmail-short_text" lang="en"><span class="gmail-">in my configuration of course.<br></span></span></div></div><div class="gmail_extra"><br></div><div class="gmail_extra"><br></div><div class="gmail_extra">Best regards<br></div><div class="gmail_extra">-- <br>Karel Ziegler<br></div></div>