[squid-users] Problem to configure squid for HTTPS website (HSTS or others certificate problems)

Tue Apr 5 00:18:42 UTC 2016

On 4/04/2016 9:25 p.m., Raph Ghost wrote:
> Hi users :)
> 
> 
> What I want to do: I have a dedicated server and I want to make it as a transparent adblocker through a VPN. So I have installed and configured OpenVPN and route my traffic from the VPN tun into the squid proxy.
> 
> 
> What is the problem: Websites based on http work great but those based on httpS doesn't work at all.
> 
> 
> I have already tried two squid configurations and look for that problem in the user mail list history but I can't find any workaround that works.
> 
> 
> My compilation options (squid 3.5.15 -with-openssl is enabled):
> 

Please upgrade to 3.5.16.

> '--with-openssl=/etc/ssl' '--enable-ssl-crtd'

 /etc is a location for config files. I somehow doubt that you have
installed the openssl binaries in there.

If you installed libssl-dev package correctly then you don't need the
"=/path" piece to be specified at all. Squid build script will find
OpenSSL in its normal place.

> My iptable conf (port 22: my ssh server/ input port 443: my OpenVPN server):
> 
> -P INPUT DROP
> -P FORWARD DROP
> -P OUTPUT DROP
> -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A INPUT -i lo -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
> -A INPUT -p tcp -m tcp --dport 53 -j ACCEPT
> -A INPUT -p udp -m udp --dport 53 -j ACCEPT
> -A INPUT -i eth0 -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT
> -A INPUT -i tun0 -j ACCEPT
> -A FORWARD -i tun0 -j ACCEPT
> -A FORWARD -o tun0 -j ACCEPT
> -A FORWARD -i tun0 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A FORWARD -i eth0 -o tun0 -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
> -A OUTPUT -o lo -j ACCEPT
> -A OUTPUT -p tcp -m tcp --dport 22 -j ACCEPT
> -A OUTPUT -p tcp -m tcp --dport 53 -j ACCEPT
> -A OUTPUT -p udp -m udp --dport 53 -j ACCEPT
> -A OUTPUT -p tcp -m tcp --dport 80 -j ACCEPT
> -A OUTPUT -p udp -m udp --dport 123 -j ACCEPT
> -A OUTPUT -o tun0 -j ACCEPT
> -A OUTPUT -p icmp -j ACCEPT
> 
> My iptable conf (nat table):
> -P PREROUTING ACCEPT
> -P INPUT ACCEPT
> -P OUTPUT ACCEPT
> -P POSTROUTING ACCEPT
> -A PREROUTING -s 10.8.0.0/24 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3129
> -A PREROUTING -s 10.8.0.0/24 -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 3130
> -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE
> 
> This iptables configuration works great to route vpn input trafic into squid.
> 
<snip>

> 2nd conf I have tried (based on many tutorials and the official squid wiki especially to configure Ssl Bump/Peek and Slice function):
> #
> # Recommended minimum configuration:
> #
<snip>

> # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
> #
> 
> # Example rule allowing access from your local networks.
> # Adapt localnet in the ACL section to list your (internal) IP networks
> # from where browsing should be allowed
> http_access allow localnet
> http_access allow localhost
> 
> # And finally deny all other access to this proxy
> http_access deny all
> 
> 
> always_direct allow all

always_direct is not relevant since 3.1. Remove.

> sslproxy_cert_error allow all

By instructing Squid to ignore all errors, you are hiding all cert
related errors. Remove the above line to see if there is some error
being encountered that is leading to your problem.

> sslproxy_cafile /etc/ssl/certs/ca-certificates.crt
> sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB
> # Squid normally listens to port 3128
> http_port 10.8.0.1:3128
> http_port 10.8.0.1:3129 transparent

"transparent" is obsolete. Use "intercept" instead.

> https_port 10.8.0.1:3130 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=6MB cert=/etc/squid/ssl_cert/myCA.pem

... 6MB here and the 4MB on the helper. Those numbers need to be the
same IIRC.

> 
> acl step1 at_step SslBump1
> acl step2 at_step SslBump2
> acl step3 at_step SslBump3
> 
> ssl_bump peek step1 all
> ssl_bump stare step2
> ssl_bump bump step3
> 
> 
> # Uncomment and adjust the following to add a disk cache directory.
> #cache_dir ufs /var/spool/squid 100 16 256
> 
> # Leave coredumps in the first cache dir
> coredump_dir /var/spool/squid
> 
> #
> # Add any of your own refresh_pattern entries above these.
> #
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
> refresh_pattern . 0 20% 4320
> 
> 
> Unfortunately none of these conf work.
> 
> With the first conf:
> If i try to connect to https://openclassrooms.com/ for example I get a warning about that the certificate is not trust. I can overpass this warning (by clicking on "continue on this website (dangerous)") but after few seconds I get an error generated by squid:
> "L'erreur suivante s'est produite en essayant d'accéder à l'URL : https://openclassrooms.com/
> La connexion 190.93.240.239 a échouée.
> Le système a retourné : (110) Connection timed out   < ----- Important line
> L'hôte distant ou le réseau sont peut-être défaillant. Veuillez renouveler votre requête.
> Votre administrateur proxy est webmaster."
> 
> In access.log I get:
> 1459756883.952     42 10.8.0.6 TCP_MISS/200 565 GET http://www.google-analytics.com/__utm.gif? - ORIGINAL_DST/216.58.214.78 image/gif
> 1459756885.636     14 10.8.0.6 TCP_MISS/204 262 GET http://www.gstatic.com/generate_204 - ORIGINAL_DST/216.58.214.67 -
> 1459756890.842     17 10.8.0.6 TCP_MISS/302 505 GET http://openclassrooms.com/ - ORIGINAL_DST/190.93.240.239 -
> 1459756891.129     14 10.8.0.6 TCP_MISS/204 262 GET http://www.gstatic.com/generate_204 - ORIGINAL_DST/216.58.214.67 -
> 1459756961.902  60814 10.8.0.6 TCP_MISS/503 4850 GET https://openclassrooms.com/ - ORIGINAL_DST/190.93.240.239 text/html
> 
> In cache.log there is nothing especial.
> 
> When I try to connect to https://www.google.fr I get a warning (from my browser, here Chrome) but I can't overpass it (due to HSTS technologie).
> 
> 
> 
> With the second conf (which is supposed to dynamically generate certificate from the original certificate to overpass HSTS - at least this I did think but it doesn't work):

When TLS is used properly/securely it cannot be bumped.

When visiting Google sites using Chrome (Google software) they employ
certificate pinning (aka hard-coded their official certificate(s) into
the browser), which prevents bumping from working.

HSTS is a different feature, which does not actually do any security
improvement to TLS itself. So can still be bumped - unless other TLS
usage prevents the bumping.

> Both of google or openclassroom websites generate the same result:
> On browser I get a ERR_TIMED_OUT.
> 
> In access.log:
> 1459755020.622  59785 10.8.0.6 TAG_NONE/200 0 CONNECT 46.228.47.114:443 - ORIGINAL_DST/46.228.47.114 -
> 1459755043.645  60448 10.8.0.6 TAG_NONE/200 0 CONNECT 46.228.47.115:443 - ORIGINAL_DST/46.228.47.115 -
> 1459755045.000  60058 10.8.0.6 TAG_NONE/200 0 CONNECT 216.58.211.106:443 - ORIGINAL_DST/216.58.211.106 -
> 
> In store.log I get SOMETIMES (rarely) this:
> 2016/04/01 11:43:05| Pinger exiting.
> 2016/04/01 11:46:02 kid1| Error negotiating SSL connection on FD 27: Closed by client
> 2016/04/01 11:46:09 kid1| Error negotiating SSL connection on FD 36: Closed by client
> 2016/04/01 11:46:16 kid1| Error negotiating SSL connection on FD 30: Closed by client
> 2016/04/01 11:46:23 kid1| Error negotiating SSL connection on FD 38: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol (1/-1)

Meaning that non-HTTPS is being sent over port 443. Sadly that is very
common these days. You will need the on_unsupported_protocol feature of
Squid-4 to get around that.

Squid-4 is also able to handle certain types of TLS hello message that
Squid-3 will report the above message for.

> 
> Whatever configuration that I used I have import certificate into my browser correctly.
> 

"I get a warning about that the certificate is not trust" indicates that
it has *not* been imported correctly into the trusted certs.

Amos