[squid-users] leaking memory in squid 3.4.8 and 3.4.7.

[Amos Jeffries]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 1/10/2014 1:24 a.m., Victor Sudakov wrote:
> Eliezer Croitoru wrote:
>>> Attaching two cachemgr reports: right after squid restart and 
>>> several hours later (grown to 816M in SIZE).
>> I am not very good at calculating what I do not know how to yet! 
>> What I do understand is that "Cumulative allocated volume: 22.198
>> GB" means that this is what was used ever by squid for this
>> uptime. Since I am not such a good calculator and the data exists
>> already lets lets try and see the next output of: mgr:mem 
>> mgr:info
> 
> I have already restarted squid several times since I obtained the 
> reports attached to the previous mail. The mgr:mem and mgr:info
> info reports don't have to be from the same process, do they?

Yes, its preferrable that they are from the same process as close to
the same time as possible.

> 
> And is there a way to obtain mgr:info etc from the command line,
> not from a browser?

The squidclient tool can be used to pull manager reports directly and
quickly. Like so:

  squidclient mgr:info >squid_mgr_info.txt
  squidclient mgr:mem  >squid_mgr_memory.tsv

Since you expressed some confusion about reading the mem report here
are some basics that should help:

The memory report table is a TSV format. The .tsv file generated above
can be opened by most any spreadsheet tools (Excell, OpenOffice Calc,
etc). You just select "TXT formatted" or "Comma Separated Value" as
the file type then select Tab as separator (de-select comma / CSV) if
asked what the format type is. That makes it easier to view and play
with the data.

In your case with pooling disabled the relevant details are in the
section of columns titled "In Use". The "Chunked" and "Allocations
Saved" sections are garbage when pools are off.

The report is sorted by which objects have been allocated most memory.
So if you are familar or can find out what the named objects are used
for then it may (or not) be easy to spot something odd.
 eg. in your first report I saw the IdentStateData was in hundreds of
MB with 50K active requests for just 150 clients. *very* odd given
your config should make no more than 3 IDENT lookups per client.

Most objects names should give away ther meaning.

The unusual one you should expect to be first is mem_node. Which is
the name for a 4KB chunk of cache_mem and in-transit objects. There is
some bytes of meta data making it slightly bigger than 4KB, which on
some OS means the memory page uses 8KB per object without telling
Squid. This can be a hidden gotcha and why Squid sometimes uses
slightly over 2x what the mem_node column says is allocated to
cache_mem. :-(


> 
>> and "top -n 1 -b|egrep "proxy|squid|PID"" Which should give a
>> nice output of the squid process memory consumption.

Be careful noting which column is resident memory and which is virtual
memory in that output. Each helper spawned is allocated virtual memory
equal to the total resident size of its squid parent process
(including cache_mem and any shared memory usage) at the time of its
spawning. It can show very huge numbers even while Squid goes nowhere
near using any of it.

HTH
Amos
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (MingW32)

iQEcBAEBAgAGBQJUKqvAAAoJELJo5wb/XPRj2rEH/2ThUFxwWX5JgXDH/Tpp/QLc
YYROxf65XpSRMUVvyygl1AVzgZxwO3XK6HQGz/bzlRWBIQFbLYuCucWLk0RBBy2Q
4CHdS90SOML4fjCUpxTXvtbH8f6jp+MNH6ULEvmCpDtmAg1o6NQD86mO/nlddfHU
/TexsrkvGMNWluMGDi/dhUJY2XHvAbMfd3l3dGpmzmDu2uPM8LwDDV26nJ07657j
BIxG1kue+sAD0LdK+DQlegedo4zu4aCy8H355eGskhUht56dx0dMniTe0q8nxP+x
y/cuiGOEjjfHwNmLAXZbSVjdMZc4V2sk9mhyXQ7CAQ4CAD3e0VxrXqsPwLh4lwE=
=/mkc
-----END PGP SIGNATURE-----