[squid-users] Ssl-bump deep dive (intercept last post and final thoughts)

Mon Jun 1 01:46:54 UTC 2015

On Mon, 2015-06-01 at 13:00 +1200, Amos Jeffries wrote:

> On 1/06/2015 11:56 a.m., James Lay wrote:
> > So this has been REALLY good!  The tl;dr:  ssl-bumping is pretty easy
> > even with intercept, ssl-bumping with access control is a little more
> > difficult...jump to the config to skip the chit chat.
> > 
> > My goal has always been to a content filter based on url regex.  This
> > works just fine for http traffic, but is much more difficult for https
> > traffic just for the case of you may or may not know the host you're
> > going to, depending on the site/app.  I'll be real honest here....I'm
> > only doing this to protect/filter the traffic of two kids, on laptops,
> > iPhone, and Android phone, so it's a mixed bag of content and, since
> > it's just the two of them in a home environment, I get to play around
> > and see what works and what doesn't.
> > 
> > Below is a close as I can get transparent intercept ssl-bump with
> > content filtering with using a list of domains/urls with both http and
> > https.  I still have to use a list of broken sites, which are large
> > netblocks (17.0.0.0/8..Apple anyone?) because some of these I just can't
> > seem to get host/domain information during the ssl handshake.  As I
> > discovered after attempting to put this into "production", I have not
> > been able to emulate using wget or curl an https session that doesn't
> > have any SNI information, so that threw me for a loop.  TextNow is a
> > great example (I'm including a packet capture of this in this post).
> > There's no host information in the client hello....there's no host
> > information in the server hello.....buried deep in the certificate ONLY
> > is the "commonName=.*textnow.me"...that's it.  This dashed my hopes of
> > using an url_regex for access control with all https sessions.  I have
> > "%ssl::>cert_subject" in my logging, and I never did see this log in any
> > of my tests...and I tested a BUNCH of different peek/stare/splice/bump
> > cominations..so I don't think squid is actually seeing this from the
> > certificate.
> > 
> > Another challenge is getting http url_regex filtering to work with https
> > filtering.  My method of filtering means not having an "http_access
> > allow localnet", which directly conflicted with also trying to filter
> > https.  The solution was to add an acl for port 443, then http_access to
> > just allow it, as our filtering was going to happen for https further
> > down.
> > 
> > I know there's a fair amount of people who just want to plop in some
> > config files, run a few commands, and be up and running.  The below
> > configuration has two additional files it references, http_url.txt,
> > which is an a list of domains/urls (\.apple\.com for example), and the
> > aptly named broken, which is a IP list (17.0.0.0/8).  The broken list
> > should be (semi) trusted and are sites that we just can't get SNI or
> > hostname information from.  If you've created a single cert/key pair
> > from the Squid documentation, you won't need the key= line in your
> > https_port directive.  If you've followed along in my posts, you already
> > have the configure line from my previous posts.  Change the
> > commands/config to fir where your squid config and ssl_db are.  So after
> > configuring, make sure you:
> > 
> > sudo /opt/libexec/ssl_crtd -c -s /opt/var/ssl_db
> > sudo chown -R nobody /opt/var/ssl_db/
> > 
> > As I believe in a lot of logging, and actually looking at said logging,
> > below is what you can expect to see in your logs (mine logs to syslog,
> > again, change this if you log to a different file):
> > 
> > Allowed http to .apple.com in http_url.txt:
> > May 31 17:03:48 gateway (squid-1): 192.168.1.100 - -
> > [31/May/2015:17:03:48 -0600] "GET
> > http://init.ess.apple.com/WebObjects/VCInit.woa/wa/getBag? HTTP/1.1" - -
> > 200 5243 TCP_MISS:ORIGINAL_DST -
> > Denied http to symcb.com not in http_url.txt
> > May 31 17:03:48 gateway (squid-1): 192.168.1.100 - -
> > [31/May/2015:17:03:48 -0600] "GET http://sd.symcb.com/sd.crt HTTP/1.1" -
> > - 403 3618 TCP_DENIED:HIER_NONE -
> > Spliced https IP in broken.txt (google block 216.58.192.0/19)
> > May 31 17:04:34 gateway (squid-1): 192.168.1.101 - -
> > [31/May/2015:17:04:34 -0600] "CONNECT 216.58.216.138:443 HTTP/1.1" - -
> > 200 568 TCP_TUNNEL:ORIGINAL_DST peek
> > Spliced https IP in broken.txt that we got SNI or bumped site in
> > http_url.txt look exactly the same
> > May 31 17:09:45 gateway (squid-1): 192.168.1.100 - -
> > [31/May/2015:17:09:45 -0600] "CONNECT 23.222.157.21:443 HTTP/1.1"
> > init.itunes.apple.com - 200 30314 TCP_TUNNEL:ORIGINAL_DST peek
> > 
> > The only drag with the configuration is you won't see when an https
> > session is terminated when the IP/url is not in the broken.txt, or the
> > http_url.txt:
> > 
> > [17:20:53 jlay at analysis:~$] wget -d
> > --ca-certificate=/etc/ssl/certs/sslsplit.crt https://www.yahoo.com
> > Setting --ca-certificate (cacertificate) to /etc/ssl/certs/sslsplit.crt
> > DEBUG output created by Wget 1.16.1 on linux-gnu.
> > 
> > URI encoding = ‘UTF-8’
> > --2015-05-31 17:20:59--  https://www.yahoo.com/
> > Resolving www.yahoo.com (www.yahoo.com)... 206.190.36.45,
> > 206.190.36.105, 2001:4998:c:a06::2:4008
> > Caching www.yahoo.com => 206.190.36.45 206.190.36.105
> > 2001:4998:c:a06::2:4008
> > Connecting to www.yahoo.com (www.yahoo.com)|206.190.36.45|:443...
> > connected.
> > Created socket 3.
> > Releasing 0x00007fdf67eecdd0 (new refcount 1).
> > Initiating SSL handshake.
> > SSL handshake failed.
> > Closed fd 3
> > Unable to establish SSL connection.
> > 
> > May 31 17:20:59 gateway (squid-1): 192.168.1.6 - - [31/May/2015:17:20:59
> > -0600] "CONNECT 206.190.36.45:443 HTTP/1.1" www.yahoo.com - 200 0
> > TAG_NONE:ORIGINAL_DST peek 
> > 
> > Full config below:
> > ####################################
> > acl localnet src 192.168.1.0/24
> > 
> > acl SSL_ports port 443
> > acl Safe_ports port 80
> > acl Safe_ports port 443
> > 
> > acl CONNECT method CONNECT
> > 
> > acl allowed_http_sites url_regex "/opt/etc/squid/http_url.txt"
> > acl allow_https port 443
> 
> Note how SSL_ports and allow_https ACL definitions are identical apart
> from the name.
> 
> You can replace all uses of "allow_https" ACL in the rest of your config
> with "SSL_Ports".
> 
> 
> > acl broken dst "/opt/etc/squid/broken.txt"
> > 
> > http_access deny !Safe_ports
> > http_access deny CONNECT !SSL_Ports
> > 
> > http_access allow allow_https
> > http_access allow allowed_http_sites
> > http_access deny !allowed_http_sites
> > 
> > http_access deny all
> 
> Two deny lines in a row are redundant when one is "deny all".
> 
> You can drop the "deny !allowed_http_sites".
> 
> 
> > 
> > acl step1 at_step SslBump1
> > acl step2 at_step SslBump2
> > acl step3 at_step SslBump3
> > 
> > ssl_bump peek step1 broken
> > ssl_bump peek step2 broken
> > ssl_bump splice broken
> > ssl_bump peek step1 all
> > ssl_bump peek step2 all
> 
> The above lines are logically the same as:
> 
>  ssl_bump splice step3 broken
>  ssl_bump peek all
> 
> > acl allowed_https_sites ssl::server_name_regex
> > "/opt/etc/squid/http_url.txt"
> > ssl_bump bump allowed_https_sites
> > ssl_bump terminate !allowed_https_sites
> 
> If you used reject instead of terminate on this rule, you should get the
> log entries you mentioned wanting when connections are terminated.
> 
> The not logging of "terminate" connectisno is a bug. Please report to
> bugzilla so we dont forget it.
> 
> > 
> > sslproxy_cert_error allow all
> > sslproxy_capath /etc/ssl/certs
> > sslproxy_flags DONT_VERIFY_PEER 
> 
> The point of TLS is to verify the other endpoint of the connection is
> actually who they claim to be. The above config settings are a) not
> bothering to even check the claim, and b) silently ignoring whatever
> result the security check produces.
> 
> * DONT_VERIFY_PEER is a debugging option. Not for use in "production"
> systems. It should not even be used in any real operational testing,
> only to see if the peer verification failures was the source of an issue.
> 
> * sslproxy_cert_error allow all - is pure evil. The directive was only
> added so networks could continue to work during transition periods when
> a major vulnerability like CRIME/ BREECH / POODLE were discovered and
> their library started erroring out on bad bevahiour, but the remote
> sites were being laggards.
>  Theres a limited number of errors which are safe to ignore, and an even
> smaller sub-set of those which are actually needed on any given network.
> 
> 
> > sslproxy_options ALL
> 
> "ALL" enables a lot of old features and hacks in OpenSSL (export grade
> ciphers and plaintext passwords, woohoo!) that are increasingly found to
> cause security vulenrabilities.
> 
> 
> > 
> > sslcrtd_program /opt/libexec/ssl_crtd -s /opt/var/ssl_db -M 4MB
> > sslcrtd_children 5
> > 
> > http_port 3128 intercept
> > https_port 3129 intercept ssl-bump
> > cert=/opt/etc/squid/certs/sslsplit_ca_cert.pem
> > cafile=/opt/etc/squid/certs/sslsplit_ca_cert.pem
> > key=/opt/etc/squid/certs/sslsplit_ca_key.pem
> > generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
> > sslflags=NO_SESSION_REUSE
> > 
> > logformat mine %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %ssl::>sni %
> > ssl::>cert_subject %>Hs %<st %Ss:%Sh %ssl::bump_mode 
> > 
> > access_log syslog:daemon.info mine
> > 
> > refresh_pattern -i (cgi-bin|\?)	0	0%	0
> > refresh_pattern .		0	20%	4320
> > 
> > coredump_dir /opt/var
> > ##############################
> > 
> > Thanks all for being patient while I continued to post my learning and
> > all my mistakes.  If there's anything that I've missed, or if there's
> > another method for trying to accomplish what I've tried to do I'm all
> > eyes.
> > 
> > James
> > 
> > P.S. Things I'd love to see in Squid some day:
> > 
> > acl's being AND'd (http_access allow allowed_sites AND localnet)
> 
> That aready exists. The AND is implicit in the order of the ACL checks,
> so the "AND " is redundant waste of space in the config file.
> 
> eg, your above "allowed_sites AND localnet" rule is configured:
>   http_access allow allowed_sites localnet
> 
> This is why I keep pointing out how useless rules like "ssl_bump peek
> step1 all" are. So you want to peek when "step1", just peek based on
> step1, no need to check if true == true.
> 
> 
> > Full on separate http_access, https_access directives
> > 
> 
> I consider this every now and again. But HTTPS is not actually a real
> thing. It is a TLS connection delivering regular HTTP messages. By the
> time the messages are inside Squid they dont have that TLS part any more
> than clear text HTTP has TCP headers.
> 
> 
> That said, if you want to go fancy you can use the "allof" ACL type.
> This ACL type lets you write an entire sub-tree of logical ACL tests and
> give it a name. Its not quite the same though.
> 
>  acl HTTP proto HTTP
>  acl HTTPS proto HTTPS
> 
>  acl sites dstdomain ...
> 
>  acl sites2 dstdomain ...
>  acl site2paths urlpath_regex ...
> 
>  acl 443 port 443
> 
>  acl plain allof CONNECT 443
>  acl plain allof sites
>  acl plain allof sites2 site2paths
>  acl plain allof localnet
> 
>  acl decrypted allof sites
>  acl decrypted allof sites2 site2paths
>  acl decrypted allof localnet
> 
>  http_access allow HTTP plain
>  http_access allow HTTPS decrypted
>  http_access deny all
> 
> Amos
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users

Awesome, awesome, and more awesome.  Amos that you SO much for the
information...I'll make the optimizations and changes and test...I won't
bug the list with this anymore unless requested however.  I will
absolutely file the bug report on terminate.

James
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20150531/84035226/attachment-0001.html>