[squid-users] How to analyse squid memory usage

[Amos Jeffries]

On 11/05/2016 4:37 p.m., Dan Charlesworth wrote:
> Thanks Amos -
> 
> Not sure how self-explanatory the output is, though.
> 
> I’ve attached the output from a site with a 12GB server where top was showing 2.9GB allocated to squid (this is normal e.g. “the control"). But the mem output shows the allocated total as ~1GB, apparently?
> 

That ~1GB looks like correct interpretation to me.

One thing to be aware of is that some OS allocate memory in multiples of
4KB. So objects like mem_node which are just a few bytes over use 8KB -
twice as much memory as Squid is aware of. mem_node is know to hit this
problem, other objects get pooled in larger batches so Squid use tends
to self-correct that. In this case the number of mem_node allocated does
not account for more than a tiny slice of 12 vs 1 GB difference.

Another thing is that virtual memory accounting by top etc is useless as
a metric. Squid uses fork() for its child processes, each time it forks
the virtual memory "size" gets doubled. So 12GB could just mean Squid is
running 11 helpers (1GB RAM used by Squid, 100KB by each of 11 helpers
== 1GB RAM aka. 12 x 1GB virtual memory !).

 ==> I see cbdata helper_server has 12 objects (ie 12 helpers running).
Which does almost exactly account for the 1 vs 12 scale of the
difference you see between mgr:mem and top.

NP: there are also STL objects and stuff not using Squid memory pools.
So there can be some MB of difference between mgr:mem report and
external tools. But that should not grow very big.


> Maybe things will become clearer once I have a “leaky” server’s output to compare with it.
> 

Leaks show up in the mem report looking a lot like the ClientInfo line
in that one you provided. Objects allocated, all in use, and none or
very few savings.
 ClientInfo here is only some KB. So probably not an actual leak, but
thats how they usually show up. The same signature in a popular object
like mem_node or HttpRequest it would be a leak.

If you need to we also have a build option for embedding valgrind hooks
into Squid between the custom pool allocators and the general code. That
allows valgrind to be run and trace leaks in its normal way, unlike the
allocator OpenSSL got a bad rep from.

Amos