[squid-users] Your cache is running out of filedescriptors

[Eliezer Croitoru]

Squid uses the root limit and also the current environment limit.
Then current environment limit can be changed only by the root user..
So your openrc script change should apply the best fix instead of allowing root have a basic high limit.
But if someone has root privilges on the machine it doesn't matter anyway so..
Choose how you want to upper the limit.
Using the basic limits way or the openrc one.

Eliezer

----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer at ngtech.co.il



-----Original Message-----
From: squid-users [mailto:squid-users-bounces at lists.squid-cache.org] On Behalf Of Vieri
Sent: Friday, September 1, 2017 18:21
To: squid-users at lists.squid-cache.org
Subject: Re: [squid-users] Your cache is running out of filedescriptors

________________________________
From: Eliezer Croitoru <eliezer at ngtech.co.il>

>
> You will need to use:> ulimit -Hn 65535
> 
> first and after this apply the lower limit:
> ulimit -n 16384
>

> As Amos suggested, since squid almost 100% requires root privileges then you can add to the openrc or system startup 

> service\script the specific limit you want to apply in the scope of any start\restart of the service(squid).

Many thanks to both of you.

I created 01_squid.conf in /etc/security/limits.d/ with:
* hard nofile 65535
* soft nofile 16384

I then restarted squid, and haven't had any issues for the last 24+ hours.

I was hoping to change that file to:
squid hard nofile 65535
squid soft nofile 16384


However, correct me if I'm wrong, but it seems to me that you're saying that Squid adjusts the limit as "root" user, not as the squid user.

I have these main processes:

root      5690  0.0  0.0  87444  5676 ?        Ss   Aug31   0:00 /usr/sbin/squid -YC -f /etc/squid/squid.conf -n squid
squid     5694  2.9  3.3 1188628 1109564 ?     S    Aug31  55:06 (squid-1) -YC -f /etc/squid/squid.conf -n squid


So, is it preferable to use the squid user name in limits.conf's "domain" field, or should I use your method by modifying my openrc init script?

BTW my system is Gentoo, and here's what I can read in the default openrc init script:

# Maximum file descriptors squid can open is determined by:
# a basic default of N=1024
#  ... altered by ./configure --with-filedescriptors=N
#  ... overridden on production by squid.conf max_filedescriptors (if,
#  and only if, setrlimit() RLIMIT_NOFILE is able to be built+used).
# Since we do not configure hard coded # of filedescriptors anymore,
# there is no need for ulimit calls in the init script.
# Use max_filedescriptors in squid.conf instead.


... and here's the start function:

start() {
checkconfig || return 1
checkpath -d -q -m 0750 -o squid:squid /run/${SVCNAME}
ebegin "Starting ${SVCNAME} (service name ${SVCNAME//[^[:alnum:]]/})"
KRB5_KTNAME="${SQUID_KEYTAB}" /usr/sbin/squid ${SQUID_OPTS} -f /etc/squid/${SVCNAME}.conf -n ${SVCNAME//[^[:alnum:]]/}
eend $? && sleep 1
}


The thing is that if Gentoo's default hard ulimit is x then I can't just set max_filedescriptors to a value >x in squid.conf. It simply won't work. Or will it?
When squid starts up as root, can it increase via setrlimit() to whatever value is in max_filedescriptors even if ulimit -Ha shows a lower value for nofiles? 


These are the defaults on my system:

# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 127512
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 127512
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited


# ulimit -Ha
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 127512
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 4096
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) unlimited
max user processes              (-u) 127512
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited


So, if I were to use your method I guess I would need to modify the init script's start() function like this:

start() {
[...]
ulimit -Hn 65535
ulimit -n 16384
ebegin "Starting ${SVCNAME} (service name ${SVCNAME//[^[:alnum:]]/})"
KRB5_KTNAME="${SQUID_KEYTAB}" /usr/sbin/squid ${SQUID_OPTS} -f /etc/squid/${SVCNAME}.conf -n ${SVCNAME//[^[:alnum:]]/}

[...]

Vieri
_______________________________________________
squid-users mailing list
squid-users at lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users