[squid-users] Huge memory required for squid 3.5

[Alex Rousskov]

On 04/26/2017 09:35 AM, Yuri Voinov wrote:

> This is openssl issue or squid's?

AFAIK, the underlying issue (i.e., bug #4005) is mostly a Squid problem:
Squid is caching SSL contexts (instead of certificates) and does a poor
job maintaining that cache.

Earlier OpenSSL versions (that had to be used when the original code was
written) complicated solving this problem. OpenSSL v1.0.1+ added APIs
that simplify some aspects of the anticipated fix. Certain OpenSSL
aspects will continue to hurt Squid, even with OpenSSL v1.0.1, but if
you want to blame a single project (instead of both), blame Squid.


> Why sessions can't share CA's data cached in memory? shared_ptr invented
> already.

OpenSSL knew how to share things well before std::shared_ptr became
available. However, it is the responsibility of the application to tell
OpenSSL what to create from scratch and what to share. A part of the
problem is that Squid tells OpenSSL to create many large things from
scratch and then caches those large things while underestimating their
size by several(?) orders of magnitude (and probably also missing many
cache hits).

More details, including the difference between problems associated with
from-client and to-server connections, are documented in the "Memory
Usage" section of http://wiki.squid-cache.org/Features/SslBump

FWIW, we have spent a lot of resources on triaging this problem and
drafting possible solutions (in various overlapping areas), but there is
currently no sponsor to finalize and implement any of the fixes. AFAIK,
bug #4005 is stuck.

I am glad that NO_DEFAULT_CA helps mitigate some of the problems in some
environments.


HTH,

Alex.


> 26.04.2017 9:08, Amos Jeffries пишет:
>> On 26/04/17 10:53, Yuri Voinov wrote:
>>> Ok, but how NO_DEFAULT_CA should help with this?
>>
>> It prevents OpenSSL copying that 1MB into each incoming client
>> connections memory. The CAs are only useful there when you have some
>> of the global CAs as root for client certificates - in which case you
>> still only want to trust the roots you paid for service and not all of
>> them.
>>
>> Just something to try if there are huge memory issues with TLS/SSL
>> proxying. The default behaviour is fixed for Squid-4 with the config
>> options changes. But due to being a major surprise for anyone already
>> relying on global roots for client certs it remains a problem in 3.5.
>>
>> Amos
>>
>> _______________________________________________
>> squid-users mailing list
>> squid-users at lists.squid-cache.org
>> http://lists.squid-cache.org/listinfo/squid-users
> 
> 
> 
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>