[squid-users] Properly filtering http and https traffic in a transparent proxy environment

[James Lay]

Resending this with photobucket links instead of including images:

http://i290.photobucket.com/albums/ll269/DigiDemon/allowed.png
http://i290.photobucket.com/albums/ll269/DigiDemon/terminate.png


Hey All,

So....here's what I have for filtering http and https in the same
instance.  This is using iptables with -j REDIRECT lines.  Below is my
entire squid.conf, documented as well as I can:

#allow local network to connect to squid
acl localnet src 192.168.1.0/24

#safe ports are 80 and 443 in one acl, port 443 is another acl
acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 443

#allow the http CONNECT method
acl CONNECT method CONNECT

#our regex list of sites and domains that we allow ie, www\.apple\.com
and \.google\.com
acl allowed_http_sites url_regex "/opt/etc/squid/http_url.txt"

#we don't want to allow anything besides port 80 and port 443
http_access deny !Safe_ports

#we don't want CONNECT if we're not going to port 443
http_access deny CONNECT !SSL_Ports

#since we may not know the https site we're going to (ie connect direct
by IP), we must initially allow all https
http_access allow SSL_ports

#we allow http, but only sites and domains in our regex http_url.txt
list above
http_access allow allowed_http_sites

#drop any other http requests that are not in our regex http_url.txt
list above
http_access deny all

#break out the ssl_bump process by steps
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3

#look for site or domain name either by SNI in request (step1), or
server subject in certificate in response (step2)
ssl_bump peek step1 all
ssl_bump peek step2 all

#see if the server name we obtained from the previous peek's above are
in our http_url.txt list above
acl allowed_https_sites ssl::server_name_regex
"/opt/etc/squid/http_url.txt"

#if the server name is in our http_url.txt, allow it 
ssl_bump splice step3 allowed_https_sites

#if the server name is not in our http_url.txt terminate the handshake
it
ssl_bump terminate all

#cert path and allow all the ssl options
sslproxy_capath /etc/ssl/certs
sslproxy_options ALL

#standard crtdaemon options
sslcrtd_program /opt/libexec/ssl_crtd -s /opt/var/ssl_db -M 4MB
sslcrtd_children 5

#intercept 3128 for port 80, and 3129 for port 443.  Cert, cacert (these
are the same, read on the list this fixed an issue), and key, generate
ssl certs
http_port 3128 intercept
https_port 3129 intercept ssl-bump
cert=/opt/etc/squid/certs/sslsplit_ca_cert.pem
cafile=/opt/etc/squid/certs/sslsplit_ca_cert.pem
key=/opt/etc/squid/certs/sslsplit_ca_key.pem
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
sslflags=NO_SESSION_REUSE

#normal-ish log format, but we want to see the SNI, server cert subject,
and how we are bumping
logformat mine %>a %[ui %[un [%tl] "%rm %ru HTTP/%rv" %ssl::>sni %
ssl::>cert_subject %>Hs %<st %Ss:%Sh %ssl::bump_mode 

#log the above format name (mine) to syslog
access_log syslog:daemon.info mine

refresh_pattern -i (cgi-bin|\?) 0 0% 0
refresh_pattern . 0 20% 4320

coredump_dir /opt/var 

One caveat I've seen so far is that I have only seen is I only ever see
peek in the logs, though I know it's been spliced as shown below:

[09:35:19 jlay at gateway:/opt/etc/squid$] grep textnow http_url.txt 
api\.textnow\.me

[09:34:57 jlay at gateway:~$] dig api.textnow.me
<snip>
;; ANSWER SECTION:
api.textnow.me. 600 IN A 209.59.180.48

>From the client:
[09:34:27 jlay at analysis:~/dev/squid$] openssl s_client -connect
209.59.180.48:443
CONNECTED(00000003)
depth=1 C = US, ST = Arizona, L = Scottsdale, O = "GoDaddy.com, Inc.",
OU = http://certificates.godaddy.com/repository, CN = Go Daddy Secure
Certification Authority, serialNumber = 07969287
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
0 s:/OU=Domain Control Validated/CN=*.textnow.me
<snip>

GET / HTTP/1.1

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 11 Jun 2015 15:36:54 GMT
Content-Type: text/plain
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.5.16
Expires: Thu, 01 Jan 1970 00:00:00 GMT
Cache-Control: no-cache, no-store, must-revalidate, max-age=0

Server Squid log entry:
Jun 11 09:36:55 gateway (squid-1): 192.168.1.6 - - [11/Jun/2015:09:36:55
-0600] "CONNECT 209.59.180.48:443 HTTP/1.1" - - 200 364
TCP_TUNNEL:ORIGINAL_DST peek

I also notice that the CN does not show up in the logs...it WAS spliced
however because it matches our http_url.txt.  From the above, it appears
that only the step1 is getting logged.   The below log entry was used
with wget (sends SNI by default):
Jun 11 08:51:05 gateway squid: 192.168.1.6 - - [11/Jun/2015:08:51:05
-0600] "CONNECT 23.211.252.28:443 HTTP/1.1" www.apple.com - 200 14388
TCP_TUNNEL:ORIGINAL_DST peek

The above shows that I logged and retrieved my SNI at step1....the
subsequent splice was not logged.  I still note that terminates do not
get logged ( I have a bug open for that but I think the core bug may be
that when using atStep you only see step1 regardless, but I could be
wrong).  I'm enclosing an image of what a https terminate looks like
from the server and the client (msn.com isn't in the http_url.txt), and
also what an https allow looks like (apple.com is in the list).

That's it.  I can verify that the above works for a single list of
allowed hosts(www.apple.com) and domains (google.com).  If there's
something I missed...something wrong...ways to improve...ANYTHING for
the betterment of Squid users please don't hesitate to jump in here.
Thank you. 

James


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20150611/63d56035/attachment.html>