[squid-users] Huge memory required for squid 3.5

[Nil Nik]

Hi,


Can we cleanup 'Cache' using -

SSL_flush_sessions(ctx, time(NULL));  or
SSL_flush_sessions(ctx,0);

Periodically or on reconfigure?

Nil

________________________________
From: squid-users <squid-users-bounces at lists.squid-cache.org> on behalf of Nil Nik <nil_fergi at hotmail.com>
Sent: Wednesday, May 3, 2017 11:54 AM
To: Alex Rousskov; squid-users at lists.squid-cache.org
Subject: Re: [squid-users] Huge memory required for squid 3.5

Hi,


NO_DEFAULT_CA doesn't help. Still goes in GB. Can anyone tell me area so that i can work on?


Regards,

Nil


________________________________
From: squid-users <squid-users-bounces at lists.squid-cache.org> on behalf of Alex Rousskov <rousskov at measurement-factory.com>
Sent: Wednesday, April 26, 2017 7:37 PM
To: squid-users at lists.squid-cache.org
Subject: Re: [squid-users] Huge memory required for squid 3.5

On 04/26/2017 09:35 AM, Yuri Voinov wrote:

> This is openssl issue or squid's?

AFAIK, the underlying issue (i.e., bug #4005) is mostly a Squid problem:
Squid is caching SSL contexts (instead of certificates) and does a poor
job maintaining that cache.

Earlier OpenSSL versions (that had to be used when the original code was
written) complicated solving this problem. OpenSSL v1.0.1+ added APIs
that simplify some aspects of the anticipated fix. Certain OpenSSL
aspects will continue to hurt Squid, even with OpenSSL v1.0.1, but if
you want to blame a single project (instead of both), blame Squid.


> Why sessions can't share CA's data cached in memory? shared_ptr invented
> already.

OpenSSL knew how to share things well before std::shared_ptr became
available. However, it is the responsibility of the application to tell
OpenSSL what to create from scratch and what to share. A part of the
problem is that Squid tells OpenSSL to create many large things from
scratch and then caches those large things while underestimating their
size by several(?) orders of magnitude (and probably also missing many
cache hits).

More details, including the difference between problems associated with
from-client and to-server connections, are documented in the "Memory
Usage" section of http://wiki.squid-cache.org/Features/SslBump
Features/SslBump - Squid Web Proxy Wiki<http://wiki.squid-cache.org/Features/SslBump>
wiki.squid-cache.org
Squid-in-the-middle decryption and encryption of straight CONNECT and transparently redirected SSL traffic, using configurable CA certificates.



FWIW, we have spent a lot of resources on triaging this problem and
drafting possible solutions (in various overlapping areas), but there is
currently no sponsor to finalize and implement any of the fixes. AFAIK,
bug #4005 is stuck.

I am glad that NO_DEFAULT_CA helps mitigate some of the problems in some
environments.


HTH,

Alex.


> 26.04.2017 9:08, Amos Jeffries пишет:
>> On 26/04/17 10:53, Yuri Voinov wrote:
>>> Ok, but how NO_DEFAULT_CA should help with this?
>>
>> It prevents OpenSSL copying that 1MB into each incoming client
>> connections memory. The CAs are only useful there when you have some
>> of the global CAs as root for client certificates - in which case you
>> still only want to trust the roots you paid for service and not all of
>> them.
>>
>> Just something to try if there are huge memory issues with TLS/SSL
>> proxying. The default behaviour is fixed for Squid-4 with the config
>> options changes. But due to being a major surprise for anyone already
>> relying on global roots for client certs it remains a problem in 3.5.
>>
>> Amos
>>
>> _______________________________________________
>> squid-users mailing list
>> squid-users at lists.squid-cache.org
>> http://lists.squid-cache.org/listinfo/squid-users
squid-users Info Page<http://lists.squid-cache.org/listinfo/squid-users>
lists.squid-cache.org
squid-users -- General discussion relating to Squid. The membership of this list is thousands of Squid users from around the world About squid-users


>
>
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
squid-users Info Page<http://lists.squid-cache.org/listinfo/squid-users>
lists.squid-cache.org
squid-users -- General discussion relating to Squid. The membership of this list is thousands of Squid users from around the world About squid-users


>

_______________________________________________
squid-users mailing list
squid-users at lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
squid-users Info Page<http://lists.squid-cache.org/listinfo/squid-users>
lists.squid-cache.org
squid-users -- General discussion relating to Squid. The membership of this list is thousands of Squid users from around the world About squid-users


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20170504/d9d71e34/attachment-0001.html>