<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=UTF-8">
<META NAME="GENERATOR" CONTENT="GtkHTML/4.6.6">
</HEAD>
<BODY>
On Mon, 2015-06-01 at 13:00 +1200, Amos Jeffries wrote:
<BLOCKQUOTE TYPE=CITE>
<PRE>
On 1/06/2015 11:56 a.m., James Lay wrote:
<FONT COLOR="#737373">> So this has been REALLY good! The tl;dr: ssl-bumping is pretty easy</FONT>
<FONT COLOR="#737373">> even with intercept, ssl-bumping with access control is a little more</FONT>
<FONT COLOR="#737373">> difficult...jump to the config to skip the chit chat.</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> My goal has always been to a content filter based on url regex. This</FONT>
<FONT COLOR="#737373">> works just fine for http traffic, but is much more difficult for https</FONT>
<FONT COLOR="#737373">> traffic just for the case of you may or may not know the host you're</FONT>
<FONT COLOR="#737373">> going to, depending on the site/app. I'll be real honest here....I'm</FONT>
<FONT COLOR="#737373">> only doing this to protect/filter the traffic of two kids, on laptops,</FONT>
<FONT COLOR="#737373">> iPhone, and Android phone, so it's a mixed bag of content and, since</FONT>
<FONT COLOR="#737373">> it's just the two of them in a home environment, I get to play around</FONT>
<FONT COLOR="#737373">> and see what works and what doesn't.</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> Below is a close as I can get transparent intercept ssl-bump with</FONT>
<FONT COLOR="#737373">> content filtering with using a list of domains/urls with both http and</FONT>
<FONT COLOR="#737373">> https. I still have to use a list of broken sites, which are large</FONT>
<FONT COLOR="#737373">> netblocks (17.0.0.0/8..Apple anyone?) because some of these I just can't</FONT>
<FONT COLOR="#737373">> seem to get host/domain information during the ssl handshake. As I</FONT>
<FONT COLOR="#737373">> discovered after attempting to put this into "production", I have not</FONT>
<FONT COLOR="#737373">> been able to emulate using wget or curl an https session that doesn't</FONT>
<FONT COLOR="#737373">> have any SNI information, so that threw me for a loop. TextNow is a</FONT>
<FONT COLOR="#737373">> great example (I'm including a packet capture of this in this post).</FONT>
<FONT COLOR="#737373">> There's no host information in the client hello....there's no host</FONT>
<FONT COLOR="#737373">> information in the server hello.....buried deep in the certificate ONLY</FONT>
<FONT COLOR="#737373">> is the "commonName=.*textnow.me"...that's it. This dashed my hopes of</FONT>
<FONT COLOR="#737373">> using an url_regex for access control with all https sessions. I have</FONT>
<FONT COLOR="#737373">> "%ssl::>cert_subject" in my logging, and I never did see this log in any</FONT>
<FONT COLOR="#737373">> of my tests...and I tested a BUNCH of different peek/stare/splice/bump</FONT>
<FONT COLOR="#737373">> cominations..so I don't think squid is actually seeing this from the</FONT>
<FONT COLOR="#737373">> certificate.</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> Another challenge is getting http url_regex filtering to work with https</FONT>
<FONT COLOR="#737373">> filtering. My method of filtering means not having an "http_access</FONT>
<FONT COLOR="#737373">> allow localnet", which directly conflicted with also trying to filter</FONT>
<FONT COLOR="#737373">> https. The solution was to add an acl for port 443, then http_access to</FONT>
<FONT COLOR="#737373">> just allow it, as our filtering was going to happen for https further</FONT>
<FONT COLOR="#737373">> down.</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> I know there's a fair amount of people who just want to plop in some</FONT>
<FONT COLOR="#737373">> config files, run a few commands, and be up and running. The below</FONT>
<FONT COLOR="#737373">> configuration has two additional files it references, http_url.txt,</FONT>
<FONT COLOR="#737373">> which is an a list of domains/urls (\.apple\.com for example), and the</FONT>
<FONT COLOR="#737373">> aptly named broken, which is a IP list (17.0.0.0/8). The broken list</FONT>
<FONT COLOR="#737373">> should be (semi) trusted and are sites that we just can't get SNI or</FONT>
<FONT COLOR="#737373">> hostname information from. If you've created a single cert/key pair</FONT>
<FONT COLOR="#737373">> from the Squid documentation, you won't need the key= line in your</FONT>
<FONT COLOR="#737373">> https_port directive. If you've followed along in my posts, you already</FONT>
<FONT COLOR="#737373">> have the configure line from my previous posts. Change the</FONT>
<FONT COLOR="#737373">> commands/config to fir where your squid config and ssl_db are. So after</FONT>
<FONT COLOR="#737373">> configuring, make sure you:</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> sudo /opt/libexec/ssl_crtd -c -s /opt/var/ssl_db</FONT>
<FONT COLOR="#737373">> sudo chown -R nobody /opt/var/ssl_db/</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> As I believe in a lot of logging, and actually looking at said logging,</FONT>
<FONT COLOR="#737373">> below is what you can expect to see in your logs (mine logs to syslog,</FONT>
<FONT COLOR="#737373">> again, change this if you log to a different file):</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> Allowed http to .apple.com in http_url.txt:</FONT>
<FONT COLOR="#737373">> May 31 17:03:48 gateway (squid-1): 192.168.1.100 - -</FONT>
<FONT COLOR="#737373">> [31/May/2015:17:03:48 -0600] "GET</FONT>
<FONT COLOR="#737373">> <A HREF="http://init.ess.apple.com/WebObjects/VCInit.woa/wa/getBag">http://init.ess.apple.com/WebObjects/VCInit.woa/wa/getBag</A>? HTTP/1.1" - -</FONT>
<FONT COLOR="#737373">> 200 5243 TCP_MISS:ORIGINAL_DST -</FONT>
<FONT COLOR="#737373">> Denied http to symcb.com not in http_url.txt</FONT>
<FONT COLOR="#737373">> May 31 17:03:48 gateway (squid-1): 192.168.1.100 - -</FONT>
<FONT COLOR="#737373">> [31/May/2015:17:03:48 -0600] "GET <A HREF="http://sd.symcb.com/sd.crt">http://sd.symcb.com/sd.crt</A> HTTP/1.1" -</FONT>
<FONT COLOR="#737373">> - 403 3618 TCP_DENIED:HIER_NONE -</FONT>
<FONT COLOR="#737373">> Spliced https IP in broken.txt (google block 216.58.192.0/19)</FONT>
<FONT COLOR="#737373">> May 31 17:04:34 gateway (squid-1): 192.168.1.101 - -</FONT>
<FONT COLOR="#737373">> [31/May/2015:17:04:34 -0600] "CONNECT 216.58.216.138:443 HTTP/1.1" - -</FONT>
<FONT COLOR="#737373">> 200 568 TCP_TUNNEL:ORIGINAL_DST peek</FONT>
<FONT COLOR="#737373">> Spliced https IP in broken.txt that we got SNI or bumped site in</FONT>
<FONT COLOR="#737373">> http_url.txt look exactly the same</FONT>
<FONT COLOR="#737373">> May 31 17:09:45 gateway (squid-1): 192.168.1.100 - -</FONT>
<FONT COLOR="#737373">> [31/May/2015:17:09:45 -0600] "CONNECT 23.222.157.21:443 HTTP/1.1"</FONT>
<FONT COLOR="#737373">> init.itunes.apple.com - 200 30314 TCP_TUNNEL:ORIGINAL_DST peek</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> The only drag with the configuration is you won't see when an https</FONT>
<FONT COLOR="#737373">> session is terminated when the IP/url is not in the broken.txt, or the</FONT>
<FONT COLOR="#737373">> http_url.txt:</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> [17:20:53 jlay@analysis:~$] wget -d</FONT>
<FONT COLOR="#737373">> --ca-certificate=/etc/ssl/certs/sslsplit.crt <A HREF="https://www.yahoo.com">https://www.yahoo.com</A></FONT>
<FONT COLOR="#737373">> Setting --ca-certificate (cacertificate) to /etc/ssl/certs/sslsplit.crt</FONT>
<FONT COLOR="#737373">> DEBUG output created by Wget 1.16.1 on linux-gnu.</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> URI encoding = ‘UTF-8’</FONT>
<FONT COLOR="#737373">> --2015-05-31 17:20:59-- <A HREF="https://www.yahoo.com/">https://www.yahoo.com/</A></FONT>
<FONT COLOR="#737373">> Resolving <A HREF="http://www.yahoo.com">www.yahoo.com</A> (<A HREF="http://www.yahoo.com">www.yahoo.com</A>)... 206.190.36.45,</FONT>
<FONT COLOR="#737373">> 206.190.36.105, 2001:4998:c:a06::2:4008</FONT>
<FONT COLOR="#737373">> Caching <A HREF="http://www.yahoo.com">www.yahoo.com</A> => 206.190.36.45 206.190.36.105</FONT>
<FONT COLOR="#737373">> 2001:4998:c:a06::2:4008</FONT>
<FONT COLOR="#737373">> Connecting to <A HREF="http://www.yahoo.com">www.yahoo.com</A> (<A HREF="http://www.yahoo.com">www.yahoo.com</A>)|206.190.36.45|:443...</FONT>
<FONT COLOR="#737373">> connected.</FONT>
<FONT COLOR="#737373">> Created socket 3.</FONT>
<FONT COLOR="#737373">> Releasing 0x00007fdf67eecdd0 (new refcount 1).</FONT>
<FONT COLOR="#737373">> Initiating SSL handshake.</FONT>
<FONT COLOR="#737373">> SSL handshake failed.</FONT>
<FONT COLOR="#737373">> Closed fd 3</FONT>
<FONT COLOR="#737373">> Unable to establish SSL connection.</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> May 31 17:20:59 gateway (squid-1): 192.168.1.6 - - [31/May/2015:17:20:59</FONT>
<FONT COLOR="#737373">> -0600] "CONNECT 206.190.36.45:443 HTTP/1.1" <A HREF="http://www.yahoo.com">www.yahoo.com</A> - 200 0</FONT>
<FONT COLOR="#737373">> TAG_NONE:ORIGINAL_DST peek </FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> Full config below:</FONT>
<FONT COLOR="#737373">> ####################################</FONT>
<FONT COLOR="#737373">> acl localnet src 192.168.1.0/24</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> acl SSL_ports port 443</FONT>
<FONT COLOR="#737373">> acl Safe_ports port 80</FONT>
<FONT COLOR="#737373">> acl Safe_ports port 443</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> acl CONNECT method CONNECT</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> acl allowed_http_sites url_regex "/opt/etc/squid/http_url.txt"</FONT>
<FONT COLOR="#737373">> acl allow_https port 443</FONT>
Note how SSL_ports and allow_https ACL definitions are identical apart
from the name.
You can replace all uses of "allow_https" ACL in the rest of your config
with "SSL_Ports".
<FONT COLOR="#737373">> acl broken dst "/opt/etc/squid/broken.txt"</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> http_access deny !Safe_ports</FONT>
<FONT COLOR="#737373">> http_access deny CONNECT !SSL_Ports</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> http_access allow allow_https</FONT>
<FONT COLOR="#737373">> http_access allow allowed_http_sites</FONT>
<FONT COLOR="#737373">> http_access deny !allowed_http_sites</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> http_access deny all</FONT>
Two deny lines in a row are redundant when one is "deny all".
You can drop the "deny !allowed_http_sites".
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> acl step1 at_step SslBump1</FONT>
<FONT COLOR="#737373">> acl step2 at_step SslBump2</FONT>
<FONT COLOR="#737373">> acl step3 at_step SslBump3</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> ssl_bump peek step1 broken</FONT>
<FONT COLOR="#737373">> ssl_bump peek step2 broken</FONT>
<FONT COLOR="#737373">> ssl_bump splice broken</FONT>
<FONT COLOR="#737373">> ssl_bump peek step1 all</FONT>
<FONT COLOR="#737373">> ssl_bump peek step2 all</FONT>
The above lines are logically the same as:
ssl_bump splice step3 broken
ssl_bump peek all
<FONT COLOR="#737373">> acl allowed_https_sites ssl::server_name_regex</FONT>
<FONT COLOR="#737373">> "/opt/etc/squid/http_url.txt"</FONT>
<FONT COLOR="#737373">> ssl_bump bump allowed_https_sites</FONT>
<FONT COLOR="#737373">> ssl_bump terminate !allowed_https_sites</FONT>
If you used reject instead of terminate on this rule, you should get the
log entries you mentioned wanting when connections are terminated.
The not logging of "terminate" connectisno is a bug. Please report to
bugzilla so we dont forget it.
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> sslproxy_cert_error allow all</FONT>
<FONT COLOR="#737373">> sslproxy_capath /etc/ssl/certs</FONT>
<FONT COLOR="#737373">> sslproxy_flags DONT_VERIFY_PEER </FONT>
The point of TLS is to verify the other endpoint of the connection is
actually who they claim to be. The above config settings are a) not
bothering to even check the claim, and b) silently ignoring whatever
result the security check produces.
* DONT_VERIFY_PEER is a debugging option. Not for use in "production"
systems. It should not even be used in any real operational testing,
only to see if the peer verification failures was the source of an issue.
* sslproxy_cert_error allow all - is pure evil. The directive was only
added so networks could continue to work during transition periods when
a major vulnerability like CRIME/ BREECH / POODLE were discovered and
their library started erroring out on bad bevahiour, but the remote
sites were being laggards.
Theres a limited number of errors which are safe to ignore, and an even
smaller sub-set of those which are actually needed on any given network.
<FONT COLOR="#737373">> sslproxy_options ALL</FONT>
"ALL" enables a lot of old features and hacks in OpenSSL (export grade
ciphers and plaintext passwords, woohoo!) that are increasingly found to
cause security vulenrabilities.
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> sslcrtd_program /opt/libexec/ssl_crtd -s /opt/var/ssl_db -M 4MB</FONT>
<FONT COLOR="#737373">> sslcrtd_children 5</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> http_port 3128 intercept</FONT>
<FONT COLOR="#737373">> https_port 3129 intercept ssl-bump</FONT>
<FONT COLOR="#737373">> cert=/opt/etc/squid/certs/sslsplit_ca_cert.pem</FONT>
<FONT COLOR="#737373">> cafile=/opt/etc/squid/certs/sslsplit_ca_cert.pem</FONT>
<FONT COLOR="#737373">> key=/opt/etc/squid/certs/sslsplit_ca_key.pem</FONT>
<FONT COLOR="#737373">> generate-host-certificates=on dynamic_cert_mem_cache_size=4MB</FONT>
<FONT COLOR="#737373">> sslflags=NO_SESSION_REUSE</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> logformat mine %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %ssl::>sni %</FONT>
<FONT COLOR="#737373">> ssl::>cert_subject %>Hs %<st %Ss:%Sh %ssl::bump_mode </FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> access_log syslog:daemon.info mine</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> refresh_pattern -i (cgi-bin|\?) 0 0% 0</FONT>
<FONT COLOR="#737373">> refresh_pattern . 0 20% 4320</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> coredump_dir /opt/var</FONT>
<FONT COLOR="#737373">> ##############################</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> Thanks all for being patient while I continued to post my learning and</FONT>
<FONT COLOR="#737373">> all my mistakes. If there's anything that I've missed, or if there's</FONT>
<FONT COLOR="#737373">> another method for trying to accomplish what I've tried to do I'm all</FONT>
<FONT COLOR="#737373">> eyes.</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> James</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> P.S. Things I'd love to see in Squid some day:</FONT>
<FONT COLOR="#737373">> </FONT>
<FONT COLOR="#737373">> acl's being AND'd (http_access allow allowed_sites AND localnet)</FONT>
That aready exists. The AND is implicit in the order of the ACL checks,
so the "AND " is redundant waste of space in the config file.
eg, your above "allowed_sites AND localnet" rule is configured:
http_access allow allowed_sites localnet
This is why I keep pointing out how useless rules like "ssl_bump peek
step1 all" are. So you want to peek when "step1", just peek based on
step1, no need to check if true == true.
<FONT COLOR="#737373">> Full on separate http_access, https_access directives</FONT>
<FONT COLOR="#737373">> </FONT>
I consider this every now and again. But HTTPS is not actually a real
thing. It is a TLS connection delivering regular HTTP messages. By the
time the messages are inside Squid they dont have that TLS part any more
than clear text HTTP has TCP headers.
That said, if you want to go fancy you can use the "allof" ACL type.
This ACL type lets you write an entire sub-tree of logical ACL tests and
give it a name. Its not quite the same though.
acl HTTP proto HTTP
acl HTTPS proto HTTPS
acl sites dstdomain ...
acl sites2 dstdomain ...
acl site2paths urlpath_regex ...
acl 443 port 443
acl plain allof CONNECT 443
acl plain allof sites
acl plain allof sites2 site2paths
acl plain allof localnet
acl decrypted allof sites
acl decrypted allof sites2 site2paths
acl decrypted allof localnet
http_access allow HTTP plain
http_access allow HTTPS decrypted
http_access deny all
Amos
_______________________________________________
squid-users mailing list
<A HREF="mailto:squid-users@lists.squid-cache.org">squid-users@lists.squid-cache.org</A>
<A HREF="http://lists.squid-cache.org/listinfo/squid-users">http://lists.squid-cache.org/listinfo/squid-users</A>
</PRE>
</BLOCKQUOTE>
<BR>
Awesome, awesome, and more awesome. Amos that you SO much for the information...I'll make the optimizations and changes and test...I won't bug the list with this anymore unless requested however. I will absolutely file the bug report on terminate.<BR>
<BR>
James
</BODY>
</HTML>