[squid-users] squid in container aborted on low memory server

[Amos Jeffries]

On 4/03/19 9:45 pm, George Xie wrote:
>     > On 4/03/19 5:39 pm, George Xie wrote:
>     > > hi all:
>     > >
>     > > Squid version: 3.5.23-5+deb9u1
>     > > Docker version 18.09.3, build 774a1f4
>     > > Linux instance-4 4.9.0-8-amd64 #1 SMP Debian 4.9.130-2 (2018-10-27)
>     > > x86_64 GNU/Linux
>     > >
>     > > I have the following squid config:
>     > >
>     > >
>     > >     http_port 127.0.0.1:3128 <http://127.0.0.1:3128>
>     > >     cache deny all
>     > >     access_log none
>     > >
>     > What is it exactly that you think this is doing in regards to Squid
>     > memory needs?
>     >
> 
> 
> sorry, I don't get your quest.
>  

I was asking to see what you were thinking was going on with those settings.

As Alex already pointed out the "cache deny all" does not reduce memory
needs of Squid in any way. It just makes 256MB of that RAM become
pointless allocating.

So, if you actually do not want the proxy caching anything, then
disabling the cache_mem (set it to 0 as per Alex response) would be the
best choice of action before you go any further.

Or if you *do* want caching, and were trying to disable it for testing
the memory issue. Then your test was wrong, and produces incorrect
conclusion. Just reducing cache_mem would be best for this case - set it
to a value that should reasonably fit this container and see if the
proxy runs okay.


...
>     > >
>     > > it appears that squid (or glibc) tries to allocate 392m memory,
>     which is
>     > > larger than host free memory 370m.
>     > > but I guess squid don't need that much memory, I have another
>     running
>     > > squid instance, which only uses < 200m memory.
>     > No doubt it is configured to use less memory. For example by reducing
>     > the default memory cache size.
>     >
> 
> 
> that running squid instance has the same config.
>  

Then something odd is going on between the two. They should indeed have
had the same behaviour (either work or same error).

Whatever the issue is it is being triggered by the large blocks of RAM
allocated by a default Squid. The easiest to modify is the cache_mem.


> 
>     > > the oddest thing is if I run squid on the host (also Debian 9)
>     directly,
>     > > not in the container, squid could start and run as normal.
>     > >
>     > Linux typically allows RAM over-allocation. Which works okay so
>     long as
>     > there is sufficient swap space and there is time between memory
>     usage to
>     > do the swap in/out process.
>     > Amos
> 
> 
> swap is disabled in the host server, so do in the container. 
> 
> after all, I wonder why squid would try to claim 392m memory if don't
> need that much.
> 

Squid thinks it does. All client traffic is denied being cached by that
"deny all". BUT ... there are internally generated items which also use
cache. So there is 256MB default RAM cache allocated and only those few
small things being put in it.

You could set it to '0' or to some small value and the allocation size
should go down accordingly.


That said, every bit of client traffic headed towards the proxy uses
memory of volatile amount and at peak times it may need to allocate
large blocks.

So disabling swap entirely on the server is not a great idea. It just
moves the error and shutdown to happen at peak traffic times when it is
least wanted.


Amos