[squid-users] [SPAM] Squid stops serving requests after squid -k reconfigure

Sun Mar 28 01:42:25 UTC 2021

Hey Acid,

Haven’t seen you here for a very long time.

The first thing is to upgrade squid if possible…

It’s better that you won’t use squid -kreconf for big blacklists.

Instead you should use some external software to match the blacklists.

The most recommended software these days is ufdbguard.

Depends on the size of your blacklist your might need to find the right solution.

The best solution would be to store the list in ram somehow.

Have you tried some kind of rbl server?

At the time I wrote some code to and some of it was merged into:

https://github.com/looterz/grimd

It has a reload url so you can update the files on disk and send a reload.

Another service I am using is:

https://github.com/andybalholm/redwood

Which has a “Classification Service” function.

It’s pretty easy to write a json http client that can run queries against this classification service.

Also you’d better use a file In the dstdomain ac ie:

acl Blacklist dstdomain “/var/blacklists/xyx.list”

http_access deny Blacklist

and inside the xyx.list file just add lines of domains like

.blacklisted-domain.com

.example.com

Etc..

All The Bests,

Eliezer

----

Eliezer Croitoru

Tech Support

Mobile: +972-5-28704261

Email:  <mailto:ngtech1ltd at gmail.com> ngtech1ltd at gmail.com

Zoom: Coming soon

From: squid-users <squid-users-bounces at lists.squid-cache.org> On Behalf Of acidflash acidflash
Sent: Saturday, March 27, 2021 10:55 AM
To: squid-users at lists.squid-cache.org
Subject: [SPAM] [squid-users] Squid stops serving requests after squid -k reconfigure

I have gone through the forums, and I haven't found an answer to the question, although it has been asked more than once.

I am running squid 3.5.X on Centos 7, the compile options are:
"configure options:  '--build=x86_64-redhat-linux-gnu' '--host=x86_64-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr' '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin' '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include' '--libdir=/usr/lib64' '--libexecdir=/usr/libexec' '--sharedstatedir=/var/lib' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--disable-strict-error-checking' '--exec_prefix=/usr' '--libexecdir=/usr/lib64/squid' '--localstatedir=/var' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--with-logdir=$(localstatedir)/log/squid' '--with-pidfile=$(localstatedir)/run/squid.pid' '--disable-dependency-tracking' '--enable-eui' '--enable-follow-x-forwarded-for' '--enable-auth' '--enable-auth-basic=DB,LDAP,MSNT-multi-domain,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB,SMB_LM,getpwnam' '--enable-auth-ntlm=smb_lm,fake' '--enable-auth-digest=file,LDAP,eDirectory' '--enable-auth-negotiate=kerberos' '--enable-external-acl-helpers=file_userip,LDAP_group,time_quota,session,unix_group,wbinfo_group,kerberos_ldap_group' '--enable-cache-digests' '--enable-cachemgr-hostname=localhost' '--enable-delay-pools' '--enable-epoll' '--enable-ident-lookups' '--enable-linux-netfilter' '--enable-removal-policies=heap,lru' '--enable-snmp' '--enable-ssl-crtd' '--enable-storeio=aufs,diskd,rock,ufs' '--enable-wccpv2' '--enable-esi' '--enable-ecap' '--with-aio' '--with-default-user=squid' '--with-dl' '--with-openssl' '--with-pthreads' '--disable-arch-native' 'build_alias=x86_64-redhat-linux-gnu' 'host_alias=x86_64-redhat-linux-gnu' 'CFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -fpie' 'LDFLAGS=-Wl,-z,relro  -pie -Wl,-z,relro -Wl,-z,now' 'CXXFLAGS=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic -fpie' 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'"

I have a service which adds domains to a blacklist file, and then calls squid -k reconfigure.
Instead of writing to the file, this service updates the file completely with new rules,
by deleting the old file, and creating a new one in its place, and then calling squid -k reconfigure.
After doing this, on odd occasions, squid will stop serving traffic completely,
until you do a squid stop, and squid start. After shutting down squid,
and starting squid up with the same rules, squid will continue to work normally.
Its probably worth mentioning that during the time that these events are taking place,
the server is under quite a bit of load, and clients don't stop sending requests via the server.
What these directives look like:

acl Porn dstdomain .xnxx.com .sex.com 
acl Drugs dstdomain .drugs.com .silkroad.eu 
http_access deny Porn
http_access deny Drugs

This also seems to be amplified when there are several squid workers (child processes) running.
In regards to order, these ACL's are above any other ACL's in the list. We have a very basic squid conf file that looks like this:
## START OF FILE
http_port 3128
cache deny all
#
access_log /var/log/squid/access.log 
cache_store_log none
cache_log /dev/null
logfile_rotate 4
#
auth_param basic program /usr/lib64/squid/basic_db_auth --dsn "DBI:mysql:host=XX.XX.XX.XX;port=XXXX;database=XXXXX" --user XXXXXX --password XXXXXXXX --plaintext --persist
# 
acl db-auth proxy_auth REQUIRED
auth_param basic credentialsttl 2 hours
auth_param basic casesensitive on
#
connect_timeout 55 minutes
#
request_header_access Allow allow all
request_header_access Authorization allow all
request_header_access WWW-Authenticate allow all
request_header_access Proxy-Authorization allow all
request_header_access Proxy-Authenticate allow all
request_header_access Cache-Control allow all
request_header_access Content-Encoding allow all
request_header_access Content-Length allow all
request_header_access Content-Type allow all
request_header_access Date allow all
request_header_access Expires allow all
request_header_access Host allow all
request_header_access If-Modified-Since allow all
request_header_access Last-Modified allow all
request_header_access Location allow all
request_header_access Pragma allow all
request_header_access Accept allow all
request_header_access Accept-Charset allow all
request_header_access Accept-Encoding allow all
request_header_access Accept-Language allow all
request_header_access Content-Language allow all
request_header_access Mime-Version allow all
request_header_access Retry-After allow all
request_header_access Title allow all
request_header_access Connection allow all
request_header_access Proxy-Connection allow all
request_header_access User-Agent allow all
request_header_access Cookie allow all
request_header_access All deny all
dns_v4_first on
via off
forwarded_for off
follow_x_forwarded_for deny all
dns_nameservers 8.8.8.8 8.8.4.4
## END OF FILE

Your help is greatly appreciated, maybe there has been some insight into this issue after 10+ years since the last time it was asked.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20210328/7a6ffaf9/attachment-0001.htm>