[squid-users] SSL intercept in explicit mode
Yuri
yvoinov at gmail.com
Tue Mar 13 17:44:41 UTC 2018
AFAIK,
SSL bump subsystem uses OpenSSL memory routines. So, first of all, most
probably leaks (if any) can be OpenSSL-related, but not squid itself.
Now let's see your config snippets.
13.03.2018 23:00, Aaron Turner пишет:
> "Usually misconfiguration leads to memory overhead."
>
> This may be true, but if you look in the list archives a few months
> ago I basically chased my tail in circles and nobody could tell me
> what I was doing wrong and so many of the docs are so old that they're
> worse then useless, they seem to suggest the wrong thing.
>
> It was literally leaking GB's worth of RAM. I even disabled all
> caching and process sizes were growing into the GB's. Turn off
> ssl-bump and the leak went away.
>
> This is what I was using:
>
> http_port 10.0.0.1:3128 ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=400MB cert=/etc/squid/ssl_cert/myCA.pem
> sslflags=NO_DEFAULT_CA
> http_port localhost:3128
> ssl_bump bump all
bump all is useless without peek/splice.
Let's see on my config snippets:
https_port 3127 intercept ssl-bump generate-host-certificates=on
dynamic_cert_mem_cache_size=10MB cert=/usr/local/squid/etc/rootCA2.crt
key=/usr/local/squid/etc/rootCA2.key
tls-cafile=/usr/local/squid/etc/rootCA12.crt
options=SINGLE_DH_USE:SINGLE_ECDH_USE
tls-dh=secp384r1:/usr/local/squid/etc/dhparam.pem
cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS
tls-no-npn sslflags=NO_DEFAULT_CA:VERIFY_CRL_ALL
http_port 3128 ssl-bump generate-host-certificates=on
dynamic_cert_mem_cache_size=10MB cert=/usr/local/squid/etc/rootCA2.crt
key=/usr/local/squid/etc/rootCA2.key
tls-cafile=/usr/local/squid/etc/rootCA12.crt
options=SINGLE_DH_USE:SINGLE_ECDH_USE
tls-dh=secp384r1:/usr/local/squid/etc/dhparam.pem
cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS
tls-no-npn sslflags=NO_DEFAULT_CA:VERIFY_CRL_ALL
tls_outgoing_options cafile=/usr/local/squid/etc/ca-bundle.crt
cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS
# Cert database on ramdisk
sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s
/ramdisk1/ssl_db -M 1GB
sslcrtd_children 32 startup=10 idle=5
# SSL bump rules
acl DiscoverSNIHost at_step SslBump1
acl NoSSLIntercept ssl::server_name_regex
"/usr/local/squid/etc/acl.url.nobump"
ssl_bump peek DiscoverSNIHost
ssl_bump splice NoSSLIntercept
ssl_bump bump all
>
> sslcrtd_program /usr/lib64/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB
> sslcrtd_children 32 startup=2 idle=2
This is defaults. Pay attention, -M is limits use ssl_db directory to 4
Mb in size. It's too few for production servers. My ramdisk for ssl db
is 1+ Gb in size:
/dev/ramdisk/ramdisk1 961M 14M 890M 2% /ramdisk1/ssl_db
> sslproxy_session_cache_size 100 MB
This is disbalanced size instead of previous setting. Why so big?
# TAG: sslproxy_session_cache_size
# Sets the cache size to use for ssl session
#Default:
# sslproxy_session_cache_size 2 MB
> sslproxy_cert_error allow all
> sslproxy_flags DONT_VERIFY_PEER
*NEVER use this options. It is unsafe.
SSL Bump is dangerous enough itself. Don't do it more unsafe
additionally by yourself.
*
>
> This was on a machine (EC2 VM) with 14GB of RAM.
Pay attention on several places:
1. OS memory allocator.
2. OpenSSL version.
3. OS configuration (IPC, shared memory, swap - all memory related).
4. Squid's memory/pools configuration.
Don't forget about: Often memory fragmentation seems like leaks. But no
leaks occurs indeed.
Also, don't forget - squid's memory consumption is not only cache_mem,
but also caching on-disk metadata (swap.state), pools settings, working
memory areas, processes memory. And - also - such things like content
adaptation (did you know wide uses ecap gzip adapter is leaky itself?).
But this is just for example.
In any case, dig to the OpenSSL/OS side. Squid's memory in most cases is ok.
I know, this appears SSL Bump is leaky. But this is not correct.
>
> --
> Aaron Turner
> https://synfin.net/ Twitter: @synfinatic
> My father once told me that respect for the truth comes close to being
> the basis for all morality. "Something cannot emerge from nothing,"
> he said. This is profound thinking if you understand how unstable
> "the truth" can be. -- Frank Herbert, Dune
>
>
> On Tue, Mar 13, 2018 at 9:47 AM, Yuri <yvoinov at gmail.com> wrote:
>> I've used it on all versions starting from 3.4.
>>
>> Now I'm using Squid 5.0.0.
>>
>> I'm afraid, my config is completely useless, because of it contains tons
>> of optimizations/tweaks/tricks and designed for customized Squid 5.0.0,
>> with different memory allocator for custom infrastructure.
>>
>> You can't just take my config, implement it and hope it will give same
>> results for you.
>>
>> At least, it uses non-system CA bundle, platform-specific configuration
>> parameters combinations, etc.
>>
>> I can say, than SSL Bump is not directly related to memory leaks. Squid
>> itself almost not contains memory leaks now. Usually misconfiguration
>> leads to memory overhead.
>>
>> As a recommendation, I can give some advices.
>>
>> 1. Use server with enough RAM. 4 Gb usually enough just for default
>> squid configuration. Usually whole system RAM usage should never be
>> bigger than 1/2 of overall physical RAM. (I.e. at least 1/3 of RAM
>> should always be free during normal running. This prevents OS allocator
>> pressure to your proxy and, also, increasing performance of proxy). In
>> case of medium proxy server 16 Gb of RAM seems big enough, but never try
>> to fill it up completely.
>>
>> 2. Don't set giant cache_mem. Remember how you platform allocates whole
>> RAM - kernel, anon pages, fs caches, etc. - and use reasonable squid's
>> memory-related settings.
>>
>> 3. Use sslflags=NO_DEFAULT_CA with your SSL Bump ports.
>>
>> 4. Never remember - SSL Bump increases your cache memory pressure due to
>> increasing caching. So, you still require to have enough memory in your
>> system.
>>
>>
>> 13.03.2018 22:25, Aaron Turner пишет:
>>> What version are you using Yuri? Can you share your config?
>>> Everytime I use ssl bump, I have massive memory leaks. It's been
>>> effectively unusable for me.
>>> --
>>> Aaron Turner
>>> https://synfin.net/ Twitter: @synfinatic
>>> My father once told me that respect for the truth comes close to being
>>> the basis for all morality. "Something cannot emerge from nothing,"
>>> he said. This is profound thinking if you understand how unstable
>>> "the truth" can be. -- Frank Herbert, Dune
>>>
>>>
>>> On Tue, Mar 13, 2018 at 9:10 AM, Yuri <yvoinov at gmail.com> wrote:
>>>> Moreover,
>>>>
>>>> SSL Bump combines with interception/explicit proxy in one setup.
>>>>
>>>> And works perfectly.
>>>>
>>>>
>>>> 13.03.2018 21:14, Marcus Kool пишет:
>>>>> "SSL bump" is the name of a complex Squid feature.
>>>>> With ssl_bump ACLs one can decide which domains can be 'spliced' (go
>>>>> through the proxy untouched) or can be 'bumped' (decrypted).
>>>>>
>>>>> Interception is not a requirement for SSL bump.
>>>>>
>>>>> Marcus
>>>>>
>>>>> On 13/03/18 11:44, Danilo V wrote:
>>>>>> I mean SSL bump in explicit mode.
>>>>>> So intercept is a essencial requirement for running SSL bump?
>>>>>>
>>>>>> Em ter, 13 de mar de 2018 às 11:10, Matus UHLAR - fantomas
>>>>>> <uhlar at fantomas.sk <mailto:uhlar at fantomas.sk>> escreveu:
>>>>>>
>>>>>> On 13.03.18 13:44, Danilo V wrote:
>>>>>> >Is it possible/feasible to configure squid in explicit mode
>>>>>> with ssl
>>>>>> >intercept?
>>>>>>
>>>>>> explicit is not intercept, intercept is not explicit.
>>>>>>
>>>>>> explicit is where browser is configured (manually or
>>>>>> automatically via WPAD)
>>>>>> to use the proxy.
>>>>>>
>>>>>> intercept is where network device forcifully redirects http/https
>>>>>> connections
>>>>>> to the proxy.
>>>>>>
>>>>>> maybe you mean SSL bump in explicit mode?
>>>>>>
>>>>>> >Due to architecture of my network it is not possible to implement
>>>>>> >transparent proxy.
>>>>>>
>>>>>> excuse me?
>>>>>> by "transparent" people mean what we usually call "intercept".
>>>>>>
>>>>>> >What would be the behavior of applications that dont support
>>>>>> proxy - i.e.
>>>>>> >dont forward requests to proxy?
>>>>>>
>>>>>> they mest be intercepted.
>>>>>>
>>>>>> --
>>>>>> Matus UHLAR - fantomas, uhlar at fantomas.sk
>>>>>> <mailto:uhlar at fantomas.sk> ; http://www.fantomas.sk/
>>>>>> Warning: I wish NOT to receive e-mail advertising to this address.
>>>>>> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
>>>>>> Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...
>>>>>> _______________________________________________
>>>>>> squid-users mailing list
>>>>>> squid-users at lists.squid-cache.org
>>>>>> <mailto:squid-users at lists.squid-cache.org>
>>>>>> http://lists.squid-cache.org/listinfo/squid-users
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> squid-users mailing list
>>>>>> squid-users at lists.squid-cache.org
>>>>>> http://lists.squid-cache.org/listinfo/squid-users
>>>>>>
>>>>> _______________________________________________
>>>>> squid-users mailing list
>>>>> squid-users at lists.squid-cache.org
>>>>> http://lists.squid-cache.org/listinfo/squid-users
>>>> --
>>>> "C++ seems like a language suitable for firing other people's legs."
>>>>
>>>> *****************************
>>>> * C++20 : Bug to the future *
>>>> *****************************
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> squid-users mailing list
>>>> squid-users at lists.squid-cache.org
>>>> http://lists.squid-cache.org/listinfo/squid-users
>>>>
>> --
>> "C++ seems like a language suitable for firing other people's legs."
>>
>> *****************************
>> * C++20 : Bug to the future *
>> *****************************
>>
>>
--
"C++ seems like a language suitable for firing other people's legs."
*****************************
* C++20 : Bug to the future *
*****************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20180313/d0312fc9/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: OpenPGP digital signature
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20180313/d0312fc9/attachment-0001.sig>
More information about the squid-users
mailing list