[squid-users] SSL intercept in explicit mode
Yuri
yvoinov at gmail.com
Tue Mar 13 17:50:38 UTC 2018
FInally,
just take a look:
This is SSL Bump-aware setup. Seems no memory leaks, yes? Normal memory
distribution.
Let's see on overall OS memory:
No leaks.
13.03.2018 23:44, Yuri пишет:
>
> AFAIK,
>
> SSL bump subsystem uses OpenSSL memory routines. So, first of all,
> most probably leaks (if any) can be OpenSSL-related, but not squid itself.
>
> Now let's see your config snippets.
>
> 13.03.2018 23:00, Aaron Turner пишет:
>> "Usually misconfiguration leads to memory overhead."
>>
>> This may be true, but if you look in the list archives a few months
>> ago I basically chased my tail in circles and nobody could tell me
>> what I was doing wrong and so many of the docs are so old that they're
>> worse then useless, they seem to suggest the wrong thing.
>>
>> It was literally leaking GB's worth of RAM. I even disabled all
>> caching and process sizes were growing into the GB's. Turn off
>> ssl-bump and the leak went away.
>>
>> This is what I was using:
>>
>> http_port 10.0.0.1:3128 ssl-bump generate-host-certificates=on
>> dynamic_cert_mem_cache_size=400MB cert=/etc/squid/ssl_cert/myCA.pem
>> sslflags=NO_DEFAULT_CA
>> http_port localhost:3128
>> ssl_bump bump all
> bump all is useless without peek/splice.
>
> Let's see on my config snippets:
>
> https_port 3127 intercept ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=10MB cert=/usr/local/squid/etc/rootCA2.crt
> key=/usr/local/squid/etc/rootCA2.key
> tls-cafile=/usr/local/squid/etc/rootCA12.crt
> options=SINGLE_DH_USE:SINGLE_ECDH_USE
> tls-dh=secp384r1:/usr/local/squid/etc/dhparam.pem
> cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS
> tls-no-npn sslflags=NO_DEFAULT_CA:VERIFY_CRL_ALL
> http_port 3128 ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=10MB cert=/usr/local/squid/etc/rootCA2.crt
> key=/usr/local/squid/etc/rootCA2.key
> tls-cafile=/usr/local/squid/etc/rootCA12.crt
> options=SINGLE_DH_USE:SINGLE_ECDH_USE
> tls-dh=secp384r1:/usr/local/squid/etc/dhparam.pem
> cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS
> tls-no-npn sslflags=NO_DEFAULT_CA:VERIFY_CRL_ALL
> tls_outgoing_options cafile=/usr/local/squid/etc/ca-bundle.crt
> cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS
> # Cert database on ramdisk
> sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s
> /ramdisk1/ssl_db -M 1GB
> sslcrtd_children 32 startup=10 idle=5
>
> # SSL bump rules
> acl DiscoverSNIHost at_step SslBump1
> acl NoSSLIntercept ssl::server_name_regex
> "/usr/local/squid/etc/acl.url.nobump"
> ssl_bump peek DiscoverSNIHost
> ssl_bump splice NoSSLIntercept
> ssl_bump bump all
>
>> sslcrtd_program /usr/lib64/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB
>> sslcrtd_children 32 startup=2 idle=2
> This is defaults. Pay attention, -M is limits use ssl_db directory to
> 4 Mb in size. It's too few for production servers. My ramdisk for ssl
> db is 1+ Gb in size:
>
> /dev/ramdisk/ramdisk1 961M 14M 890M 2% /ramdisk1/ssl_db
>
>> sslproxy_session_cache_size 100 MB
> This is disbalanced size instead of previous setting. Why so big?
>
> # TAG: sslproxy_session_cache_size
> # Sets the cache size to use for ssl session
> #Default:
> # sslproxy_session_cache_size 2 MB
>
>> sslproxy_cert_error allow all
>> sslproxy_flags DONT_VERIFY_PEER
> *NEVER use this options. It is unsafe.
>
> SSL Bump is dangerous enough itself. Don't do it more unsafe
> additionally by yourself.
> *
>> This was on a machine (EC2 VM) with 14GB of RAM.
> Pay attention on several places:
>
> 1. OS memory allocator.
> 2. OpenSSL version.
> 3. OS configuration (IPC, shared memory, swap - all memory related).
> 4. Squid's memory/pools configuration.
>
> Don't forget about: Often memory fragmentation seems like leaks. But
> no leaks occurs indeed.
>
> Also, don't forget - squid's memory consumption is not only cache_mem,
> but also caching on-disk metadata (swap.state), pools settings,
> working memory areas, processes memory. And - also - such things like
> content adaptation (did you know wide uses ecap gzip adapter is leaky
> itself?).
>
> But this is just for example.
>
> In any case, dig to the OpenSSL/OS side. Squid's memory in most cases
> is ok.
>
> I know, this appears SSL Bump is leaky. But this is not correct.
>> --
>> Aaron Turner
>> https://synfin.net/ Twitter: @synfinatic
>> My father once told me that respect for the truth comes close to being
>> the basis for all morality. "Something cannot emerge from nothing,"
>> he said. This is profound thinking if you understand how unstable
>> "the truth" can be. -- Frank Herbert, Dune
>>
>>
>> On Tue, Mar 13, 2018 at 9:47 AM, Yuri <yvoinov at gmail.com> wrote:
>>> I've used it on all versions starting from 3.4.
>>>
>>> Now I'm using Squid 5.0.0.
>>>
>>> I'm afraid, my config is completely useless, because of it contains tons
>>> of optimizations/tweaks/tricks and designed for customized Squid 5.0.0,
>>> with different memory allocator for custom infrastructure.
>>>
>>> You can't just take my config, implement it and hope it will give same
>>> results for you.
>>>
>>> At least, it uses non-system CA bundle, platform-specific configuration
>>> parameters combinations, etc.
>>>
>>> I can say, than SSL Bump is not directly related to memory leaks. Squid
>>> itself almost not contains memory leaks now. Usually misconfiguration
>>> leads to memory overhead.
>>>
>>> As a recommendation, I can give some advices.
>>>
>>> 1. Use server with enough RAM. 4 Gb usually enough just for default
>>> squid configuration. Usually whole system RAM usage should never be
>>> bigger than 1/2 of overall physical RAM. (I.e. at least 1/3 of RAM
>>> should always be free during normal running. This prevents OS allocator
>>> pressure to your proxy and, also, increasing performance of proxy). In
>>> case of medium proxy server 16 Gb of RAM seems big enough, but never try
>>> to fill it up completely.
>>>
>>> 2. Don't set giant cache_mem. Remember how you platform allocates whole
>>> RAM - kernel, anon pages, fs caches, etc. - and use reasonable squid's
>>> memory-related settings.
>>>
>>> 3. Use sslflags=NO_DEFAULT_CA with your SSL Bump ports.
>>>
>>> 4. Never remember - SSL Bump increases your cache memory pressure due to
>>> increasing caching. So, you still require to have enough memory in your
>>> system.
>>>
>>>
>>> 13.03.2018 22:25, Aaron Turner пишет:
>>>> What version are you using Yuri? Can you share your config?
>>>> Everytime I use ssl bump, I have massive memory leaks. It's been
>>>> effectively unusable for me.
>>>> --
>>>> Aaron Turner
>>>> https://synfin.net/ Twitter: @synfinatic
>>>> My father once told me that respect for the truth comes close to being
>>>> the basis for all morality. "Something cannot emerge from nothing,"
>>>> he said. This is profound thinking if you understand how unstable
>>>> "the truth" can be. -- Frank Herbert, Dune
>>>>
>>>>
>>>> On Tue, Mar 13, 2018 at 9:10 AM, Yuri <yvoinov at gmail.com> wrote:
>>>>> Moreover,
>>>>>
>>>>> SSL Bump combines with interception/explicit proxy in one setup.
>>>>>
>>>>> And works perfectly.
>>>>>
>>>>>
>>>>> 13.03.2018 21:14, Marcus Kool пишет:
>>>>>> "SSL bump" is the name of a complex Squid feature.
>>>>>> With ssl_bump ACLs one can decide which domains can be 'spliced' (go
>>>>>> through the proxy untouched) or can be 'bumped' (decrypted).
>>>>>>
>>>>>> Interception is not a requirement for SSL bump.
>>>>>>
>>>>>> Marcus
>>>>>>
>>>>>> On 13/03/18 11:44, Danilo V wrote:
>>>>>>> I mean SSL bump in explicit mode.
>>>>>>> So intercept is a essencial requirement for running SSL bump?
>>>>>>>
>>>>>>> Em ter, 13 de mar de 2018 às 11:10, Matus UHLAR - fantomas
>>>>>>> <uhlar at fantomas.sk <mailto:uhlar at fantomas.sk>> escreveu:
>>>>>>>
>>>>>>> On 13.03.18 13:44, Danilo V wrote:
>>>>>>> >Is it possible/feasible to configure squid in explicit mode
>>>>>>> with ssl
>>>>>>> >intercept?
>>>>>>>
>>>>>>> explicit is not intercept, intercept is not explicit.
>>>>>>>
>>>>>>> explicit is where browser is configured (manually or
>>>>>>> automatically via WPAD)
>>>>>>> to use the proxy.
>>>>>>>
>>>>>>> intercept is where network device forcifully redirects http/https
>>>>>>> connections
>>>>>>> to the proxy.
>>>>>>>
>>>>>>> maybe you mean SSL bump in explicit mode?
>>>>>>>
>>>>>>> >Due to architecture of my network it is not possible to implement
>>>>>>> >transparent proxy.
>>>>>>>
>>>>>>> excuse me?
>>>>>>> by "transparent" people mean what we usually call "intercept".
>>>>>>>
>>>>>>> >What would be the behavior of applications that dont support
>>>>>>> proxy - i.e.
>>>>>>> >dont forward requests to proxy?
>>>>>>>
>>>>>>> they mest be intercepted.
>>>>>>>
>>>>>>> --
>>>>>>> Matus UHLAR - fantomas, uhlar at fantomas.sk
>>>>>>> <mailto:uhlar at fantomas.sk> ; http://www.fantomas.sk/
>>>>>>> Warning: I wish NOT to receive e-mail advertising to this address.
>>>>>>> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
>>>>>>> Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...
>>>>>>> _______________________________________________
>>>>>>> squid-users mailing list
>>>>>>> squid-users at lists.squid-cache.org
>>>>>>> <mailto:squid-users at lists.squid-cache.org>
>>>>>>> http://lists.squid-cache.org/listinfo/squid-users
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> squid-users mailing list
>>>>>>> squid-users at lists.squid-cache.org
>>>>>>> http://lists.squid-cache.org/listinfo/squid-users
>>>>>>>
>>>>>> _______________________________________________
>>>>>> squid-users mailing list
>>>>>> squid-users at lists.squid-cache.org
>>>>>> http://lists.squid-cache.org/listinfo/squid-users
>>>>> --
>>>>> "C++ seems like a language suitable for firing other people's legs."
>>>>>
>>>>> *****************************
>>>>> * C++20 : Bug to the future *
>>>>> *****************************
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> squid-users mailing list
>>>>> squid-users at lists.squid-cache.org
>>>>> http://lists.squid-cache.org/listinfo/squid-users
>>>>>
>>> --
>>> "C++ seems like a language suitable for firing other people's legs."
>>>
>>> *****************************
>>> * C++20 : Bug to the future *
>>> *****************************
>>>
>>>
>
> --
> "C++ seems like a language suitable for firing other people's legs."
>
> *****************************
> * C++20 : Bug to the future *
> *****************************
--
"C++ seems like a language suitable for firing other people's legs."
*****************************
* C++20 : Bug to the future *
*****************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20180313/a1d34dfc/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gchdeocgbabplcep.png
Type: image/png
Size: 20149 bytes
Desc: not available
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20180313/a1d34dfc/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ncdmadnlnjklmdpe.png
Type: image/png
Size: 17141 bytes
Desc: not available
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20180313/a1d34dfc/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: OpenPGP digital signature
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20180313/a1d34dfc/attachment-0001.sig>
More information about the squid-users
mailing list