[squid-users] Problem to configure squid for HTTPS website (HSTS or others certificate problems)
Raph Ghost
raphghost22 at hotmail.com
Mon Apr 4 09:25:41 UTC 2016
Hi users :)
What I want to do: I have a dedicated server and I want to make it as a transparent adblocker through a VPN. So I have installed and configured OpenVPN and route my traffic from the VPN tun into the squid proxy.
What is the problem: Websites based on http work great but those based on httpS doesn't work at all.
I have already tried two squid configurations and look for that problem in the user mail list history but I can't find any workaround that works.
My compilation options (squid 3.5.15 -with-openssl is enabled):
'--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--libexecdir=${prefix}/lib/squid3' '--srcdir=.' '--disable-maintainer-mode' '--disable-dependency-tracking' '--disable-silent-rules' 'BUILDCXXFLAGS=-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -fPIE -pie -Wl,-z,relro -Wl,-z,now' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--libexecdir=/usr/lib/squid' '--mandir=/usr/share/man' '--enable-inline' '--disable-arch-native' '--enable-async-io=8' '--enable-storeio=ufs,aufs,diskd,rock' '--enable-removal-policies=lru,heap' '--enable-delay-pools' '--enable-cache-digests' '--enable-icap-client' '--enable-follow-x-forwarded-for' '--enable-auth-basic=DB,fake,getpwnam,LDAP,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB' '--enable-auth-digest=file,LDAP' '--enable-auth-negotiate=kerberos,wrapper' '--enable-auth-ntlm=fake,smb_lm' '--enable-external-acl-helpers=file_userip,kerberos_ldap_group,LDAP_group,session,SQL_session,unix_group,wbinfo_group' '--enable-url-rewrite-helpers=fake' '--enable-eui' '--enable-esi' '--enable-icmp' '--enable-zph-qos' '--enable-ecap' '--disable-translation' '--with-openssl=/etc/ssl' '--enable-ssl-crtd' '--with-swapdir=/var/spool/squid' '--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid' '--with-filedescriptors=65536' '--with-large-files' '--with-default-user=proxy' '--enable-build-info= linux' '--enable-linux-netfilter' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wall' 'LDFLAGS=-fPIE -pie -Wl,-z,relro -Wl,-z,now' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security'
My iptable conf (port 22: my ssh server/ input port 443: my OpenVPN server):
-P INPUT DROP
-P FORWARD DROP
-P OUTPUT DROP
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 53 -j ACCEPT
-A INPUT -p udp -m udp --dport 53 -j ACCEPT
-A INPUT -i eth0 -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT
-A INPUT -i tun0 -j ACCEPT
-A FORWARD -i tun0 -j ACCEPT
-A FORWARD -o tun0 -j ACCEPT
-A FORWARD -i tun0 -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i eth0 -o tun0 -m state --state RELATED,ESTABLISHED -j ACCEPT
-A OUTPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A OUTPUT -o lo -j ACCEPT
-A OUTPUT -p tcp -m tcp --dport 22 -j ACCEPT
-A OUTPUT -p tcp -m tcp --dport 53 -j ACCEPT
-A OUTPUT -p udp -m udp --dport 53 -j ACCEPT
-A OUTPUT -p tcp -m tcp --dport 80 -j ACCEPT
-A OUTPUT -p udp -m udp --dport 123 -j ACCEPT
-A OUTPUT -o tun0 -j ACCEPT
-A OUTPUT -p icmp -j ACCEPT
My iptable conf (nat table):
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-A PREROUTING -s 10.8.0.0/24 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3129
-A PREROUTING -s 10.8.0.0/24 -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 3130
-A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE
This iptables configuration works great to route vpn input trafic into squid.
1st squid conf I've tried (based on this tutorial http://monblog.system-linux.net/blog/2013/03/23/mettre-en-oeuvre-squid-transparent-https-sous-debian/ and others):
#
# Recommended minimum configuration:
#
# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 10.8.0.0/24 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access deny !Safe_ports
# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports
# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager
# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost
#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost
# And finally deny all other access to this proxy
http_access deny all
# Squid normally listens to port 3128
http_port 10.8.0.1:3128
http_port 10.8.0.1:3129 transparent
https_port 10.8.0.1:3130 intercept ssl-bump key=/etc/squid/ssl_cert2/key.key cert=/etc/squid/ssl_cert2/cert.pem
# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /var/spool/squid 100 16 256
# Leave coredumps in the first cache dir
coredump_dir /var/spool/squid
#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
2nd conf I have tried (based on many tutorials and the official squid wiki especially to configure Ssl Bump/Peek and Slice function):
#
# Recommended minimum configuration:
#
# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 10.8.0.0/24 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access deny !Safe_ports
# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports
# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager
# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost
#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost
# And finally deny all other access to this proxy
http_access deny all
always_direct allow all
sslproxy_cert_error allow all
sslproxy_cafile /etc/ssl/certs/ca-certificates.crt
sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB
# Squid normally listens to port 3128
http_port 10.8.0.1:3128
http_port 10.8.0.1:3129 transparent
https_port 10.8.0.1:3130 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=6MB cert=/etc/squid/ssl_cert/myCA.pem
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3
ssl_bump peek step1 all
ssl_bump stare step2
ssl_bump bump step3
# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /var/spool/squid 100 16 256
# Leave coredumps in the first cache dir
coredump_dir /var/spool/squid
#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
Unfortunately none of these conf work.
With the first conf:
If i try to connect to https://openclassrooms.com/ for example I get a warning about that the certificate is not trust. I can overpass this warning (by clicking on "continue on this website (dangerous)") but after few seconds I get an error generated by squid:
"L'erreur suivante s'est produite en essayant d'accéder à l'URL : https://openclassrooms.com/
La connexion 190.93.240.239 a échouée.
Le système a retourné : (110) Connection timed out < ----- Important line
L'hôte distant ou le réseau sont peut-être défaillant. Veuillez renouveler votre requête.
Votre administrateur proxy est webmaster."
In access.log I get:
1459756883.952 42 10.8.0.6 TCP_MISS/200 565 GET http://www.google-analytics.com/__utm.gif? - ORIGINAL_DST/216.58.214.78 image/gif
1459756885.636 14 10.8.0.6 TCP_MISS/204 262 GET http://www.gstatic.com/generate_204 - ORIGINAL_DST/216.58.214.67 -
1459756890.842 17 10.8.0.6 TCP_MISS/302 505 GET http://openclassrooms.com/ - ORIGINAL_DST/190.93.240.239 -
1459756891.129 14 10.8.0.6 TCP_MISS/204 262 GET http://www.gstatic.com/generate_204 - ORIGINAL_DST/216.58.214.67 -
1459756961.902 60814 10.8.0.6 TCP_MISS/503 4850 GET https://openclassrooms.com/ - ORIGINAL_DST/190.93.240.239 text/html
In cache.log there is nothing especial.
When I try to connect to https://www.google.fr I get a warning (from my browser, here Chrome) but I can't overpass it (due to HSTS technologie).
With the second conf (which is supposed to dynamically generate certificate from the original certificate to overpass HSTS - at least this I did think but it doesn't work):
Both of google or openclassroom websites generate the same result:
On browser I get a ERR_TIMED_OUT.
In access.log:
1459755020.622 59785 10.8.0.6 TAG_NONE/200 0 CONNECT 46.228.47.114:443 - ORIGINAL_DST/46.228.47.114 -
1459755043.645 60448 10.8.0.6 TAG_NONE/200 0 CONNECT 46.228.47.115:443 - ORIGINAL_DST/46.228.47.115 -
1459755045.000 60058 10.8.0.6 TAG_NONE/200 0 CONNECT 216.58.211.106:443 - ORIGINAL_DST/216.58.211.106 -
In store.log I get SOMETIMES (rarely) this:
2016/04/01 11:43:05| Pinger exiting.
2016/04/01 11:46:02 kid1| Error negotiating SSL connection on FD 27: Closed by client
2016/04/01 11:46:09 kid1| Error negotiating SSL connection on FD 36: Closed by client
2016/04/01 11:46:16 kid1| Error negotiating SSL connection on FD 30: Closed by client
2016/04/01 11:46:23 kid1| Error negotiating SSL connection on FD 38: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol (1/-1)
2016/04/01 11:46:31 kid1| Error negotiating SSL connection on FD 18: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol (1/-1)
2016/04/01 11:46:38 kid1| Error negotiating SSL connection on FD 43: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol (1/-1)
2016/04/01 11:46:46 kid1| Error negotiating SSL connection on FD 10: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol (1/-1)
2016/04/01 11:46:54 kid1| Error negotiating SSL connection on FD 33: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol (1/-1)
2016/04/01 11:47:32 kid1| Error negotiating SSL connection on FD 14: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol (1/-1)
2016/04/01 11:47:40 kid1| Error negotiating SSL connection on FD 28: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol (1/-1)
2016/04/01 11:47:48 kid1| Error negotiating SSL connection on FD 10: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol (1/-1)
2016/04/01 11:47:55 kid1| Error negotiating SSL connection on FD 16: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol (1/-1)
2016/04/01 11:48:03 kid1| Error negotiating SSL connection on FD 27: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol (1/-1)
2016/04/01 11:48:54 kid1| Error negotiating SSL connection on FD 10: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol (1/-1)
2016/04/01 11:49:01 kid1| Error negotiating SSL connection on FD 14: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol (1/-1)
2016/04/01 11:49:08 kid1| Error negotiating SSL connection on FD 18: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol (1/-1)
2016/04/01 11:49:15 kid1| Error negotiating SSL connection on FD 28: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol (1/-1)
2016/04/01 11:49:24 kid1| Error negotiating SSL connection on FD 36: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol (1/-1)
2016/04/01 11:49:31 kid1| Error negotiating SSL connection on FD 38: Closed by client
2016/04/01 11:49:38 kid1| Error negotiating SSL connection on FD 40: Closed by client
2016/04/01 11:49:45 kid1| Error negotiating SSL connection on FD 53: Closed by client
2016/04/01 11:49:53 kid1| Error negotiating SSL connection on FD 59: error:14076102:SSL routines:SSL23_GET_CLIENT_HELLO:unsupported protocol (1/-1)
2016/04/01 12:22:55 kid1| Logfile: opening log stdio:/var/log/squid/netdb.state
2016/04/01 12:22:55 kid1| Logfile: closing log stdio:/var/log/squid/netdb.state
2016/04/01 12:22:55 kid1| NETDB state saved; 4 entries, 0 msec
2016/04/01 13:04:35 kid1| Logfile: opening log stdio:/var/log/squid/netdb.state
2016/04/01 13:04:35 kid1| Logfile: closing log stdio:/var/log/squid/netdb.state
2016/04/01 13:04:35 kid1| NETDB state saved; 4 entries, 0 msec
2016/04/01 13:54:40 kid1| Logfile: opening log stdio:/var/log/squid/netdb.state
2016/04/01 13:54:40 kid1| Logfile: closing log stdio:/var/log/squid/netdb.state
Whatever configuration that I used I have import certificate into my browser correctly.
Can you help me ??
Thanks in advance and sorry if my English is not perfect. I can send you any other informations that you would need.
Best regards,
Raph
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20160404/2c256780/attachment-0001.html>
More information about the squid-users
mailing list