[squid-users] [SPAM] Squid stops serving requests after squid -k reconfigure

Tue Mar 30 20:08:02 UTC 2021

Hey Acid,

First you should try 4.x latest.
a reload of 50 domains every 10 second doesn't make sense.
I don't understand the config and the setup.
for 50 sites you just need a basic script and even one in bash will work
dor you with grep.
I wrote an example in ruby a while ago, I will try to find it in the next
week.
Maybe the server is overloaded.

I will try to give you an example later on.

Eliezer

בתאריך יום ג׳, 30 במרץ 2021, 20:41, מאת acidflash acidflash ‏<
acidflash_ at linuxmail.org>:

>
> Hi Eliezer,
>
> I hope your doing ok. Thanks for the reply. Yeah currently what I am doing
> is:
>
> include /etc/squid/blockedsites.list
> and adding the ACL's and the denies in the list file. What version do you
> recommend I upgrade to, and is this a known issue? The list is actually
> pretty small, probably no more than 50 sites or so, and thats split across
> 4 or 5 groups (different ACL's). I'll look into ufdbguard and the other
> projects as well, does this sound familiar though? If you think that the
> best path forward is to alleviate the burden off of squid to some external
> tool, I could probably think up a few hacks to for that, but would
> obviously prefer to keep it all within squid. Is this occurance common with
> squid -k reconfigure and dstdomain matching? Thanks for your time. Stay
> safe.
>
> *Sent:* Sunday, March 28, 2021 at 4:42 AM
> *From:* "Eliezer Croitoru" <ngtech1ltd at gmail.com>
> *To:* squid-users at lists.squid-cache.org
> *Subject:* Re: [squid-users] [SPAM] Squid stops serving requests after
> squid -k reconfigure
>
> Hey Acid,
>
>
>
> Haven’t seen you here for a very long time.
>
> The first thing is to upgrade squid if possible…
>
>
>
> It’s better that you won’t use squid -kreconf for big blacklists.
>
> Instead you should use some external software to match the blacklists.
>
> The most recommended software these days is ufdbguard.
>
> Depends on the size of your blacklist your might need to find the right
> solution.
>
> The best solution would be to store the list in ram somehow.
>
> Have you tried some kind of rbl server?
>
>
>
> At the time I wrote some code to and some of it was merged into:
>
> https://github.com/looterz/grimd
>
>
>
> It has a reload url so you can update the files on disk and send a reload.
>
>
>
> Another service I am using is:
>
> https://github.com/andybalholm/redwood
>
>
>
> Which has a “Classification Service” function.
>
> It’s pretty easy to write a json http client that can run queries against
> this classification service.
>
>
>
> Also you’d better use a file In the dstdomain ac ie:
>
> acl Blacklist dstdomain “/var/blacklists/xyx.list”
>
> http_access deny Blacklist
>
>
>
> and inside the xyx.list file just add lines of domains like
>
> .blacklisted-domain.com
>
> .example.com
>
>
>
> Etc..
>
>
>
>
>
> All The Bests,
>
> Eliezer
>
>
>
> ----
>
> Eliezer Croitoru
>
> Tech Support
>
> Mobile: +972-5-28704261
>
> Email: ngtech1ltd at gmail.com
>
> Zoom: Coming soon
>
>
>
>
>
> *From:* squid-users <squid-users-bounces at lists.squid-cache.org> *On
> Behalf Of *acidflash acidflash
> *Sent:* Saturday, March 27, 2021 10:55 AM
> *To:* squid-users at lists.squid-cache.org
> *Subject:* [SPAM] [squid-users] Squid stops serving requests after squid
> -k reconfigure
>
>
>
> I have gone through the forums, and I haven't found an answer to the
> question, although it has been asked more than once.
>
> I am running squid 3.5.X on Centos 7, the compile options are:
> "configure options:  '--build=x86_64-redhat-linux-gnu'
> '--host=x86_64-redhat-linux-gnu' '--program-prefix=' '--prefix=/usr'
> '--exec-prefix=/usr' '--bindir=/usr/bin' '--sbindir=/usr/sbin'
> '--sysconfdir=/etc' '--datadir=/usr/share' '--includedir=/usr/include'
> '--libdir=/usr/lib64' '--libexecdir=/usr/libexec'
> '--sharedstatedir=/var/lib' '--mandir=/usr/share/man'
> '--infodir=/usr/share/info' '--disable-strict-error-checking'
> '--exec_prefix=/usr' '--libexecdir=/usr/lib64/squid' '--localstatedir=/var'
> '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid'
> '--with-logdir=$(localstatedir)/log/squid'
> '--with-pidfile=$(localstatedir)/run/squid.pid'
> '--disable-dependency-tracking' '--enable-eui'
> '--enable-follow-x-forwarded-for' '--enable-auth'
> '--enable-auth-basic=DB,LDAP,MSNT-multi-domain,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB,SMB_LM,getpwnam'
> '--enable-auth-ntlm=smb_lm,fake'
> '--enable-auth-digest=file,LDAP,eDirectory'
> '--enable-auth-negotiate=kerberos'
> '--enable-external-acl-helpers=file_userip,LDAP_group,time_quota,session,unix_group,wbinfo_group,kerberos_ldap_group'
> '--enable-cache-digests' '--enable-cachemgr-hostname=localhost'
> '--enable-delay-pools' '--enable-epoll' '--enable-ident-lookups'
> '--enable-linux-netfilter' '--enable-removal-policies=heap,lru'
> '--enable-snmp' '--enable-ssl-crtd' '--enable-storeio=aufs,diskd,rock,ufs'
> '--enable-wccpv2' '--enable-esi' '--enable-ecap' '--with-aio'
> '--with-default-user=squid' '--with-dl' '--with-openssl' '--with-pthreads'
> '--disable-arch-native' 'build_alias=x86_64-redhat-linux-gnu'
> 'host_alias=x86_64-redhat-linux-gnu' 'CFLAGS=-O2 -g -pipe -Wall
> -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong
> --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -mtune=generic
> -fpie' 'LDFLAGS=-Wl,-z,relro  -pie -Wl,-z,relro -Wl,-z,now' 'CXXFLAGS=-O2
> -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
> -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches
> -m64 -mtune=generic -fpie'
> 'PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig'"
>
>
>
> I have a service which adds domains to a blacklist file, and then calls
> squid -k reconfigure.
> Instead of writing to the file, this service updates the file completely
> with new rules,
> by deleting the old file, and creating a new one in its place, and then
> calling squid -k reconfigure.
> After doing this, on odd occasions, squid will stop serving traffic
> completely,
> until you do a squid stop, and squid start. After shutting down squid,
> and starting squid up with the same rules, squid will continue to work
> normally.
> Its probably worth mentioning that during the time that these events are
> taking place,
> the server is under quite a bit of load, and clients don't stop sending
> requests via the server.
> What these directives look like:
>
> acl Porn dstdomain .xnxx.com .sex.com
> acl Drugs dstdomain .drugs.com .silkroad.eu
> http_access deny Porn
> http_access deny Drugs
>
>
>
> This also seems to be amplified when there are several squid workers
> (child processes) running.
> In regards to order, these ACL's are above any other ACL's in the list. We
> have a very basic squid conf file that looks like this:
> ## START OF FILE
> http_port 3128
> cache deny all
> #
> access_log /var/log/squid/access.log
> cache_store_log none
> cache_log /dev/null
> logfile_rotate 4
> #
> auth_param basic program /usr/lib64/squid/basic_db_auth --dsn
> "DBI:mysql:host=XX.XX.XX.XX;port=XXXX;database=XXXXX" --user XXXXXX
> --password XXXXXXXX --plaintext --persist
> #
> acl db-auth proxy_auth REQUIRED
> auth_param basic credentialsttl 2 hours
> auth_param basic casesensitive on
> #
> connect_timeout 55 minutes
> #
> request_header_access Allow allow all
> request_header_access Authorization allow all
> request_header_access WWW-Authenticate allow all
> request_header_access Proxy-Authorization allow all
> request_header_access Proxy-Authenticate allow all
> request_header_access Cache-Control allow all
> request_header_access Content-Encoding allow all
> request_header_access Content-Length allow all
> request_header_access Content-Type allow all
> request_header_access Date allow all
> request_header_access Expires allow all
> request_header_access Host allow all
> request_header_access If-Modified-Since allow all
> request_header_access Last-Modified allow all
> request_header_access Location allow all
> request_header_access Pragma allow all
> request_header_access Accept allow all
> request_header_access Accept-Charset allow all
> request_header_access Accept-Encoding allow all
> request_header_access Accept-Language allow all
> request_header_access Content-Language allow all
> request_header_access Mime-Version allow all
> request_header_access Retry-After allow all
> request_header_access Title allow all
> request_header_access Connection allow all
> request_header_access Proxy-Connection allow all
> request_header_access User-Agent allow all
> request_header_access Cookie allow all
> request_header_access All deny all
> dns_v4_first on
> via off
> forwarded_for off
> follow_x_forwarded_for deny all
> dns_nameservers 8.8.8.8 8.8.4.4
> ## END OF FILE
>
>
>
> Your help is greatly appreciated, maybe there has been some insight into
> this issue after 10+ years since the last time it was asked.
>
>
> _______________________________________________ squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20210330/f185a701/attachment-0001.htm>