[squid-users] High CPU-Usage with squid 3.4.9 (and/or 3.4.4)

[Marcus Kool]

>> during our last tests (with 3.4.x) we also tried the worker
>> option. it does not matter if workers are enabled or not. with more
>> workers the cpu rise seems to be somewhat slower. so it is not
>> connected to (smp)workers. it is the external auth helper -
>> although the squid process and not the helper does consume all the
>> cpu...
>
> The only difference between SMP and non-SMP mode here is that non-SMP
> has 1 worker doing all the work with one CPU core, whereas SMP mode
> has several workers. They can all hit the same issues independently
> for the same reason(s).
>
>
> I am of the understanding that the code associated with the helper
> processe is using a lot of CPU doing *something* that consumes a lot
> of cycles.
>
> There is a bunch of code doing cache lookups on previous helper
> queries, queueing new lookups, generating and parsing strings in the
> I/O, and even sometimes running whole trees of ACL logics when the
> helper(s) respond.
>   So to get anywhere on this complaint it is important to know what
> (from the above set of things) exactly the CPU is doing.

Indeed but setting debug_options to ALL,9 does not work since the log file
already is too big and unmanageable even before Squid begins to do
thing that consumes CPU time.

I have a script for a daemon that I wrote. The script is executes when
the daemon receives a fatal signal (e.g. SIGSEGV):  the daemon catches
SIGSEV and executes the script which saves a stack trace (using gdb)
of all threads in a file, and then finally the daemon exits.  It is a
nice debug tool.
Maybe we can make a similar script for Squid that does something
like this:
collect the pids of all processes and then for all pids run gdb
that attaches to the process, dumps a stack trace and detaches.
The script can even do it 25 times to have a better insight in what
squid does.
An administrator can then run the script at the time that CPU peaks
and send the output to the Squid developers.
Do you like the idea?

Another idea is to implement a user-defined signal handler, say
on receipt of SIGUSR2, squid sets the debug_options to ALL,9
without rereading the config file, and after 3 seconds sets
the debug_options back to the configured value.
This way you get a 3-second sample of what Squid is doing at a
specific time and the log file will have a reasonable size.

Marcus

> Amos
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.22 (MingW32)
>
> iQEcBAEBAgAGBQJUYODRAAoJELJo5wb/XPRjvrkH/RBEitKpvLcWypNPCCrMTtw6
> tKftf9gz4og6GOkUiipE0qNLTMgWvV7Fk9/By3vhEGNL+WpQG5UEWCbfpc2h2RdL
> H5tIWJGnXcsV1PGwYI1cuyDpanNs6EnSvKnSGTZ2DdWabiFEOPr9FR/8QtVqpUdS
> EK3uMpnmZ0mbo7auDIPxwa7CYh44tC/C3VMZSto+peB1ikiDonU9B0tXVEFCheeE
> B0IWs8FaoYByVd54lL6cYPz7HcOtyt2Hb6uyPJyQVrrEJs2JuI4ZQh0X7B2mbzAi
> HK8wBbDcyC4ZKagq4ABQIYHsxwqxiNFD6v9ntXBjZpORG1opXLMSBAh9K0Ycq5s=
> =5pNQ
> -----END PGP SIGNATURE-----
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>