[squid-users] Squid memory leak on ubuntu 14.04

[风声]

We did not enable squid cache, so I think memory is ok for our case, and we
run squid servers (without cache, without cache cluster, just as forward
proxy) more than 100 servers more than 1 years in AWS serveral regions with
EC2 c3.xlarge on ubuntu 12.04. It was always running well.

Just after upgrade ubuntu 14.04, we found the memory usage increased.

Server Spec: AWS EC2 c3.xlarge (4 Cores, 7.5GB Memory, 2 x 40 GB SSD)

Before upgrade:

12.04:
Memory usage is always less than 50% (3.5GB), will increase or decrease
because traffic changes
CPU is very low, same as Disk IO, B/W (In or Out) is 500Mb/s at most, is
around 200Mb/s most of time.

14:04
Memory usage is about 80-90 % (nearly 7GB), will increase , but it decrease
very slow, and always keeping more than 50% (3.5GB),
CPU is very low, same as Disk IO, B/W (In or Out) is 500Mb/s at most, is
around 200Mb/s most of time.

I tested with squid-3.3.8 (ubuntu offical packages), and squid-3.5.11 on
12.04 and 14.04, I think it is most likely ubuntu related issue ? because
same version, same configs, but different OS versions.



2015-11-30 11:37 GMT+08:00 Amos Jeffries <squid3 at treenet.co.nz>:

> On 30/11/2015 3:19 p.m., 风声 wrote:
> > I try to use jemalloc, but from monitoring, there is no difference,
> >
> > I follow this guide:
> > https://github.com/jemalloc/jemalloc/wiki/Getting-Started
> >
> > I used LD_PRELOAD to let squid use jemalloc.
> >
> > is there something wrong ?
> >
> > If I want to re-compile squid with jemalloc, how can i do that ? Can I
> just
> > use some FLAGS ?
> >
>
> A number of build variables can be set on the command line to
> ./configure.  See the end of the "./configure --help" for the ones your
> Squid supports.
>
> But, hold up a minute there.
>
> You have provided all sorts of snapshots of what the system thinks
> memory usage is. But not told us anything about what values you expect
> to see there, or what the machine has available. Just a blank statement
> that its "high".
>
> I see that the *entire machine* is using ~8GB of RAM. For those of us
> managing machines with 32-128 GB of RAM thats actually pretty low. For
> machines with only a few hundred MB of RAM plugged in thats impossible
> numbers. So please provide some context about the machiens limits.
>
>
> Also, be aware that "Virtual Memory" is just that *virtual*. It is not
> real in-use memory. Whenever fork() is usesd to spawn a child helper or
> pworker process the current memory value of the parent process is
> assigned to the child - regardless of whether it needs or uses it.
> Effectively doubling the "Virtual Memory" numbers with every helper
> started, even when the helper uses only a few KB.
>
>
> I see from the netstat output that you have ~6318 sockets currently in
> play. Thats up to ~1.6 GB of RAM just for connection buffering right there.
>
> Then whatever the cache requirements are. Reaching several GB of RAM in
> actual usage is pretty easy.
>
> Then only after that is all added together multiply by however many
> dozen or hundred helpers are being run, and thats what the virtual
> memory numbers can say Squid is "using". Though in reality it is not
> even close.
>
> Amos
>
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20151130/31d57a80/attachment-0001.html>