[squid-users] SSL intercept in explicit mode
Aaron Turner
synfinatic at gmail.com
Tue Mar 13 19:15:05 UTC 2018
Thanks Yuri. That helps. As for the "sslproxy_flags
DONT_VERIFY_PEER", yes I understand the risks. In my specific case,
where my "users" are actually a bunch of automated web clients doing
some web crawling it's the right thing to do.
--
Aaron Turner
https://synfin.net/ Twitter: @synfinatic
My father once told me that respect for the truth comes close to being
the basis for all morality. "Something cannot emerge from nothing,"
he said. This is profound thinking if you understand how unstable
"the truth" can be. -- Frank Herbert, Dune
On Tue, Mar 13, 2018 at 10:44 AM, Yuri <yvoinov at gmail.com> wrote:
> AFAIK,
>
> SSL bump subsystem uses OpenSSL memory routines. So, first of all, most
> probably leaks (if any) can be OpenSSL-related, but not squid itself.
>
> Now let's see your config snippets.
>
> 13.03.2018 23:00, Aaron Turner пишет:
>
> "Usually misconfiguration leads to memory overhead."
>
> This may be true, but if you look in the list archives a few months
> ago I basically chased my tail in circles and nobody could tell me
> what I was doing wrong and so many of the docs are so old that they're
> worse then useless, they seem to suggest the wrong thing.
>
> It was literally leaking GB's worth of RAM. I even disabled all
> caching and process sizes were growing into the GB's. Turn off
> ssl-bump and the leak went away.
>
> This is what I was using:
>
> http_port 10.0.0.1:3128 ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=400MB cert=/etc/squid/ssl_cert/myCA.pem
> sslflags=NO_DEFAULT_CA
> http_port localhost:3128
> ssl_bump bump all
>
> bump all is useless without peek/splice.
>
> Let's see on my config snippets:
>
> https_port 3127 intercept ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=10MB cert=/usr/local/squid/etc/rootCA2.crt
> key=/usr/local/squid/etc/rootCA2.key
> tls-cafile=/usr/local/squid/etc/rootCA12.crt
> options=SINGLE_DH_USE:SINGLE_ECDH_USE
> tls-dh=secp384r1:/usr/local/squid/etc/dhparam.pem
> cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS
> tls-no-npn sslflags=NO_DEFAULT_CA:VERIFY_CRL_ALL
> http_port 3128 ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=10MB cert=/usr/local/squid/etc/rootCA2.crt
> key=/usr/local/squid/etc/rootCA2.key
> tls-cafile=/usr/local/squid/etc/rootCA12.crt
> options=SINGLE_DH_USE:SINGLE_ECDH_USE
> tls-dh=secp384r1:/usr/local/squid/etc/dhparam.pem
> cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS
> tls-no-npn sslflags=NO_DEFAULT_CA:VERIFY_CRL_ALL
> tls_outgoing_options cafile=/usr/local/squid/etc/ca-bundle.crt
> cipher=HIGH:MEDIUM:RC4:3DES:!aNULL:!eNULL:!LOW:!MD5:!EXP:!PSK:!SRP:!DSS
> # Cert database on ramdisk
> sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s
> /ramdisk1/ssl_db -M 1GB
> sslcrtd_children 32 startup=10 idle=5
>
> # SSL bump rules
> acl DiscoverSNIHost at_step SslBump1
> acl NoSSLIntercept ssl::server_name_regex
> "/usr/local/squid/etc/acl.url.nobump"
> ssl_bump peek DiscoverSNIHost
> ssl_bump splice NoSSLIntercept
> ssl_bump bump all
>
>
> sslcrtd_program /usr/lib64/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB
> sslcrtd_children 32 startup=2 idle=2
>
> This is defaults. Pay attention, -M is limits use ssl_db directory to 4 Mb
> in size. It's too few for production servers. My ramdisk for ssl db is 1+ Gb
> in size:
>
> /dev/ramdisk/ramdisk1 961M 14M 890M 2% /ramdisk1/ssl_db
>
> sslproxy_session_cache_size 100 MB
>
> This is disbalanced size instead of previous setting. Why so big?
>
> # TAG: sslproxy_session_cache_size
> # Sets the cache size to use for ssl session
> #Default:
> # sslproxy_session_cache_size 2 MB
>
> sslproxy_cert_error allow all
> sslproxy_flags DONT_VERIFY_PEER
>
> NEVER use this options. It is unsafe.
>
> SSL Bump is dangerous enough itself. Don't do it more unsafe additionally by
> yourself.
>
> This was on a machine (EC2 VM) with 14GB of RAM.
>
> Pay attention on several places:
>
> 1. OS memory allocator.
> 2. OpenSSL version.
> 3. OS configuration (IPC, shared memory, swap - all memory related).
> 4. Squid's memory/pools configuration.
>
> Don't forget about: Often memory fragmentation seems like leaks. But no
> leaks occurs indeed.
>
> Also, don't forget - squid's memory consumption is not only cache_mem, but
> also caching on-disk metadata (swap.state), pools settings, working memory
> areas, processes memory. And - also - such things like content adaptation
> (did you know wide uses ecap gzip adapter is leaky itself?).
>
> But this is just for example.
>
> In any case, dig to the OpenSSL/OS side. Squid's memory in most cases is ok.
>
> I know, this appears SSL Bump is leaky. But this is not correct.
>
> --
> Aaron Turner
> https://synfin.net/ Twitter: @synfinatic
> My father once told me that respect for the truth comes close to being
> the basis for all morality. "Something cannot emerge from nothing,"
> he said. This is profound thinking if you understand how unstable
> "the truth" can be. -- Frank Herbert, Dune
>
>
> On Tue, Mar 13, 2018 at 9:47 AM, Yuri <yvoinov at gmail.com> wrote:
>
> I've used it on all versions starting from 3.4.
>
> Now I'm using Squid 5.0.0.
>
> I'm afraid, my config is completely useless, because of it contains tons
> of optimizations/tweaks/tricks and designed for customized Squid 5.0.0,
> with different memory allocator for custom infrastructure.
>
> You can't just take my config, implement it and hope it will give same
> results for you.
>
> At least, it uses non-system CA bundle, platform-specific configuration
> parameters combinations, etc.
>
> I can say, than SSL Bump is not directly related to memory leaks. Squid
> itself almost not contains memory leaks now. Usually misconfiguration
> leads to memory overhead.
>
> As a recommendation, I can give some advices.
>
> 1. Use server with enough RAM. 4 Gb usually enough just for default
> squid configuration. Usually whole system RAM usage should never be
> bigger than 1/2 of overall physical RAM. (I.e. at least 1/3 of RAM
> should always be free during normal running. This prevents OS allocator
> pressure to your proxy and, also, increasing performance of proxy). In
> case of medium proxy server 16 Gb of RAM seems big enough, but never try
> to fill it up completely.
>
> 2. Don't set giant cache_mem. Remember how you platform allocates whole
> RAM - kernel, anon pages, fs caches, etc. - and use reasonable squid's
> memory-related settings.
>
> 3. Use sslflags=NO_DEFAULT_CA with your SSL Bump ports.
>
> 4. Never remember - SSL Bump increases your cache memory pressure due to
> increasing caching. So, you still require to have enough memory in your
> system.
>
>
> 13.03.2018 22:25, Aaron Turner пишет:
>
> What version are you using Yuri? Can you share your config?
> Everytime I use ssl bump, I have massive memory leaks. It's been
> effectively unusable for me.
> --
> Aaron Turner
> https://synfin.net/ Twitter: @synfinatic
> My father once told me that respect for the truth comes close to being
> the basis for all morality. "Something cannot emerge from nothing,"
> he said. This is profound thinking if you understand how unstable
> "the truth" can be. -- Frank Herbert, Dune
>
>
> On Tue, Mar 13, 2018 at 9:10 AM, Yuri <yvoinov at gmail.com> wrote:
>
> Moreover,
>
> SSL Bump combines with interception/explicit proxy in one setup.
>
> And works perfectly.
>
>
> 13.03.2018 21:14, Marcus Kool пишет:
>
> "SSL bump" is the name of a complex Squid feature.
> With ssl_bump ACLs one can decide which domains can be 'spliced' (go
> through the proxy untouched) or can be 'bumped' (decrypted).
>
> Interception is not a requirement for SSL bump.
>
> Marcus
>
> On 13/03/18 11:44, Danilo V wrote:
>
> I mean SSL bump in explicit mode.
> So intercept is a essencial requirement for running SSL bump?
>
> Em ter, 13 de mar de 2018 às 11:10, Matus UHLAR - fantomas
> <uhlar at fantomas.sk <mailto:uhlar at fantomas.sk>> escreveu:
>
> On 13.03.18 13:44, Danilo V wrote:
> >Is it possible/feasible to configure squid in explicit mode
> with ssl
> >intercept?
>
> explicit is not intercept, intercept is not explicit.
>
> explicit is where browser is configured (manually or
> automatically via WPAD)
> to use the proxy.
>
> intercept is where network device forcifully redirects http/https
> connections
> to the proxy.
>
> maybe you mean SSL bump in explicit mode?
>
> >Due to architecture of my network it is not possible to implement
> >transparent proxy.
>
> excuse me?
> by "transparent" people mean what we usually call "intercept".
>
> >What would be the behavior of applications that dont support
> proxy - i.e.
> >dont forward requests to proxy?
>
> they mest be intercepted.
>
> --
> Matus UHLAR - fantomas, uhlar at fantomas.sk
> <mailto:uhlar at fantomas.sk> ; http://www.fantomas.sk/
> Warning: I wish NOT to receive e-mail advertising to this address.
> Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
> Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> <mailto:squid-users at lists.squid-cache.org>
> http://lists.squid-cache.org/listinfo/squid-users
>
>
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
> --
> "C++ seems like a language suitable for firing other people's legs."
>
> *****************************
> * C++20 : Bug to the future *
> *****************************
>
>
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
> --
> "C++ seems like a language suitable for firing other people's legs."
>
> *****************************
> * C++20 : Bug to the future *
> *****************************
>
>
>
> --
> "C++ seems like a language suitable for firing other people's legs."
>
> *****************************
> * C++20 : Bug to the future *
> *****************************
More information about the squid-users
mailing list