[squid-users] url_rewrite_program and ACLs

Fri Nov 10 00:17:18 UTC 2017

On 10/11/17 00:39, Vieri wrote:
> 
> ________________________________
> From: Amos Jeffries <squid3 at treenet.co.nz>
>>
>> Darn. You have the one case that calls for keeping the helper :-(
>>
>> You can still move the ACLs that load in a reasonable times into
>> squid.conf and leave the others in SG/ufdbguard. Using
>> url_rewrite_access to restrict which transactions the helper gets
>> involved with. That will reduce its latency impact on lie traffic, but
>> still cause much the same memory related (non-)issues as now.
> 
> 
> That's exactly what I'm doing right now...
> Thanks.
> 
>> Running "squid -k shutdown" a _second_ time sends the running proxy a
>> signal to immediately skip to the processing as if the shutdown_lifetime
>> had already been reached.
> 
> 
> Thanks for that double-shutdown signal trick. I'll have to try that asap.
> 
> I'm making progress (sort of) on the FD (non-)issues I'm having.
> 
> I'll try to post back to Alex asap.
> 
> I have a custom perl script that does MySQL lookups for blacklisted sites (lots of them - so I can't use ACLs within squid.conf). I define that helper with external_acl_type.
> 
> Yesterday I changed my squid.conf by disabling this helper, and used squidGuard instead.
> I noticed a huge improvement.

Hmm that is suspicious. AFAIK SquidGuards' one remaining useful feature 
is how it loads large lists into memory in big chunks then processes 
after its technically already processing traffic from Squid - whereas 
Squid loads the files line by line, which is slower initially. Once 
loaded there is no difference in the lookup algorithms, and the SQL DB 
storage should be no different to how SG does it.

I would compare your custom script to the ext_sql_session_acl.pl.in 
script we bundle with current Squid.
  If yours lacks concurrency channel-ID I highly recommend adding that 
behaviour.
  If the DB is designed to store the protocol scheme, domain[:port] and 
path?query portions of URLs in separate columns it will be more 
efficient to pass those parameters as separate (%PROTO %DST %PORT %PATH) 
to the helper instead of just %URI.

The overheads in Squid of using external_acl_type helper interface 
should be slightly less than the url_rewrite_program for SG. The SQL DB 
data loading is about the same or better than what SG does AFAIK.

> 
> I took this snapshot yesterday:
> 
> 15:25 08/11/2017:
> 
> File descriptor usage for squid:
> Maximum number of file descriptors: 65536
> Largest file desc currently in use: 2730
> Number of file desc currently in use: 1838
> Files queued for open: 0
> Available number of file descriptors: 63698
> Reserved number of file descriptors: 100
> Store Disk files open: 0
> 
> Today I took another peak and found:
> 
> Thu Nov 9 12:19:05 CET 2017:
> 
> File descriptor usage for squid:
> Maximum number of file descriptors: 65536
> Largest file desc currently in use: 6980
> Number of file desc currently in use: 6627
> Files queued for open: 0
> Available number of file descriptors: 58909
> Reserved number of file descriptors: 100
> Store Disk files open: 0
> 
> The FDs are still increasing steadily, but a LOT less.
> 
> On the other hand, the "free" RAM went from 2GB yesterday to just 275MB today:
> 

Ouch, but kind of expected with those FD number increase. Each client 
connection will use about 3 FD, and two of them will use ~70KB each and 
the third will use some multiple of your avg object size.

Which reminds me ... Are you using SSL-Bump? if so ensure that you have 
configured "sslflags=NO_DEFAULT_CA" on the port lines. The default 
Trusted CA set can add a huge amount of useless memory to each client 
connection, which can add up to many GB quite quickly.

> # free --mega
> total used free shared buff/cache available
> Mem: 32865 8685 275 157 23904 23683
> Swap: 37036 286 36750
> 
> Used swap is still low enough (unchanged actually), so I guess I don't need to worry about it.

Nod, until the RAM runs out entirely, then problems are definitely to be 
expected and that sounds like it is your problem now.

FYI: The first side-effect of RAM swapping is that Squid starts slowing 
down on completed transactions - memory cache hits slow by 3-6 orders of 
magnitude when swapping in/out from disk, and any I/O buffers swapped 
out get 1+ speed penalties for the swap I/O time.
  That all leads to more active client connections overlapping their 
active periods (thus more FD usage total), and also clients start 
opening more connections to get better service from parallel fetches. So 
FD usage gets growth from two directions simultaneously.
  Which is all in a feedback loop since that extra memory pressure from 
more FD slows all transactions even further ... until the machine goes 
crunch. Maybe a literal crunch - I actually physically blew up a test 
machine (fire and smoke puring out the back!) measuring the effects of 
RAID on a overloaded proxy about a decade ago.

> 
> However, I'm bound to have issues when the "free" mem reaches 0... and I bet it will eventually.
> That's when the double-shutdown trick will kick in.
> 
> I'll review the perl helper code, or maybe just switch to ufdbGuard.
> 
> Thanks,
> 
> Vieri
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
> 