[squid-users] How to intercept ssl_bump transparent NAT https websites

Sun May 28 22:03:49 UTC 2017

On 29/05/17 07:52, Andi wrote:
> Hi
>
>     I installed Squid 3.5.25 at debian with libecap3 too.
>
>     Now my old squid.conf file for v3.48 not work anymore for
>     redirected https websites.
>     I get SSL_ERROR_RX_RECORD_TOO_LONG in Firefox.
>     I redirected them before by Shorewall and it worked with v3.48
>     #SQUID-PORTS
>     REDIRECT    loc    3140 tcp    https    - !192.168.1.254
>     REDIRECT    loc    3139 tcp    www    - !192.168.1.254
>
>
>     If I change https_port to http_port and remove the intercept
>     option for ssl_bump it works with expicit configured clients for
>     that port even for gmail website too.
>     What I need to change to make squid 3.5 work transparently  ?
>

SSL_ERROR_RX_RECORD_TOO_LONG is apparently what gets displayed if the 
response coming back from an attempted TLS/SSL connection is not TLS/SSL 
protocol. Such as Squid responding with an HTTP error message, or 
something like that happening.

Your below config has port 3128 for explicit-proxy traffic, port 3139 
for intercepted port 80 traffic, and 3140 for intercepted port 443 traffic.

Your log startup confirms that:

 > 2017/05/28 16:07:55.701| Accepting HTTP Socket connections at 
local=0.0.0.0:3138 remote=[::] FD 28 flags=9
 > 2017/05/28 16:07:55.702| Accepting NAT intercepted HTTP Socket 
connections at local=0.0.0.0:3139 remote=[::] FD 29 flags=41
 > 2017/05/28 16:07:55.702| Accepting NAT intercepted SSL bumped HTTPS 
Socket connections at local=0.0.0.0:3140 remote=[::] FD 30 flags=41

The first thing I would check is that the shorewall definitions of 
"https" and "www" are actually 80 and 443 respectively.

Then try to find out what Squid is sending to Firefox that would result 
in that particular error. I suspect either ICAP or SquidGuard is trying 
to change or produce a plan-text response to the initial CONNECT 
messages Squid uses internally for the SSL-Bump steps.

NP: the "abandoning" messages in cache.log are nothing to worry about 
when you are ssl-bump'ing with Squid-3, it is just an annoying 
side-effect of how SSL-Bump takes the connection away from the normal 
CONNECT tunnel handling code. IIRC it has been fixed in Squid-4 along 
with a lot of similar little PITA things.

PS. I've highlighted some improvements you can make to the config below. 
They are not related to your problem though.

>     squid -v
>     Squid Cache: Version 3.5.25
>     Service Name: squid
>     configure options:  '--build=x86_64-linux-gnu' '--prefix=/usr'
>     '--localstatedir=/var/squid' '--libexecdir=/lib/squid'
>     '--srcdir=.' '--datadir=/share/squid' '--sysconfdir=/etc/squid'
>     '--disable-ipv6' '--with-default-user=proxy'
>     '--with-logdir=/var/log/squid35'
>     '--with-pidfile=/var/run/squid35.pid' '--with-openssl'
>     '--enable-ssl-crtd' '--infodir=/share/info'
>     '--includedir=/include' '--mandir=/usr/share/man'
>     '--enable-inline' '--disable-arch-native'
>     '--disable-maintainer-mode' '--disable-dependency-tracking'
>     '--disable-silent-rules' '--enable-async-io=8'
>     '--enable-storeio=ufs,aufs,diskd,rock'
>     '--enable-removal-policies=lru,heap' '--enable-delay-pools'
>     '--enable-cache-digests' '--enable-icap-client'
>     '--enable-follow-x-forwarded-for'
>     '--enable-auth-basic=DB,fake,getpwnam,LDAP,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB'
>     '--enable-auth-digest=file,LDAP'
>     '--enable-auth-negotiate=kerberos,wrapper'
>     '--enable-auth-ntlm=fake,smb_lm'
>     '--enable-external-acl-helpers=file_userip,kerberos_ldap_group,LDAP_group,session,SQL_session,unix_group,wbinfo_group'
>     '--enable-url-rewrite-helpers=fake' '--enable-eui' '--enable-esi'
>     '--enable-icmp' '--enable-zph-qos' '--enable-ecap'
>     '--disable-translation' '--with-filedescriptors=65536'
>     '--with-large-files' '--enable-linux-netfilter' 'CFLAGS=-g -O2
>     -fPIE -fstack-protector-strong -Wformat -Werror=format-security
>     -Wall' 'LDFLAGS=-fPIE -pie -Wl,-z,relro -Wl,-z,now'
>     'CPPFLAGS=-D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 -fPIE
>     -fstack-protector-strong -Wformat -Werror=format-security'
>     'build_alias=x86_64-linux-gnu'
>
>     cat /etc/squid/squid.conf:
>     debug_options ALL,6
>     #0 26,2 83,2 33,2 17,2 44,2
>     logformat datetime  %tl %6tr CLIENT:%>a = = %Ss %<Hs %rm=%>ru
>     --%[un %Sh/%<a %mt
>     access_log  /var/log/squid35/access.log datetime
>     forwarded_for on
>     error_directory /usr/share/squid/errors/de-de/
>

Have you altered or otherwise touched the files in that directory?
If not I suggest using this instead:

   error_default_language de-de
<http://master.squid-cache.org/Doc/config/error_default_language/>

>     acl localnet src 192.168.1.0/24
>     acl SSL_ports port 443
>     acl Safe_ports port 80          # http
>     acl Safe_ports port 21          # ftp
>     acl Safe_ports port 443         # https
>     acl Safe_ports port 70          # gopher
>     acl Safe_ports port 210         # wais
>     acl Safe_ports port 1025-65535  # unregistered ports
>     acl Safe_ports port 280         # http-mgmt
>     acl Safe_ports port 488         # gss-http
>     acl Safe_ports port 591         # filemaker
>     acl Safe_ports port 777         # multiling http
>     acl CONNECT method CONNECT
>     http_access deny !Safe_ports
>     http_access deny CONNECT !SSL_ports
>     http_access allow localhost manager
>     http_access deny manager
>     http_access deny to_localhost
>     http_access allow localnet
>     http_access allow localhost
>     http_reply_access allow all
>

No need to explicitly "allow all" for replies. That happens anyway. That 
directive is mostly useful to deny things.

>     http_access deny all
>     icp_access allow localnet
>     icp_access deny all
>     ### NEW for v3.5x SSL-Bump ###
>     always_direct allow all
>

That "always_direct" was a hack to workaround a bug in the first 
ssl-bump code. It is long since irrelevant. I recommend removing it.

>     acl step1 at_step SslBump1
>     acl step2 at_step SslBump2
>     acl step3 at_step SslBump3
>     ssl_bump splice localhost
>     #acl exclude_sites ssl::server_name_regex -i
>     "/var/lib/squidguard/db/BL/whitelist-ssl/whitelist.destdomainlist"
>     ssl_bump peek step1 all
>     #ssl_bump splice exclude_sites
>     ssl_bump stare step2 all
>

You don't need the "all" on the above lines. The "step2 all" is both 
unnecessary and adds confusion since that line does *not* apply to all 
step2 traffic - some was spliced instead by the previous line.

>     ssl_bump bump all
>     #############################
>     http_port 0.0.0.0:3138
>     http_port 0.0.0.0:3139 intercept
>     sslproxy_cert_adapt setCommonName ssl::certDomainMismatch
>     https_port 0.0.0.0:3140 intercept ssl-bump
>     generate-host-certificates=on dynamic_cert_mem_cache_size=16MB
>     cert=/etc/squid/myca.pem
>     sslproxy_options NO_SSLv2,NO_SSLv3,SINGLE_DH_USE
>     sslproxy_capath /etc/ssl/certs
>     ##sslproxy_cafile /etc/ssl/certs/ca-certificates.crt
>     sslcrtd_program /bin/ssl_crtd -s /var/spool/squid_ssldb -M 16MB
>     sslcrtd_children 10
>     cache_dir ufs /etc/squid/ssl_db 100 16 256
>

Why are you storing all cacheable *HTTP* objects into /etc/squid/ssl_db ?
  especially since your SSL certificate store is /var/spool/squid_ssldb ?

>     cache_mgr admin at mainrouter
>     visible_hostname xxx
>     httpd_suppress_version_string on
>     coredump_dir /var/spool/squid
>     refresh_pattern ^ftp: 1440    20%     10080
>     refresh_pattern ^gopher: 1440    0%      1440
>     refresh_pattern -i (/cgi-bin/|\?) 0 0%      0
>     refresh_pattern                0       20%     4320
>     cache_effective_user proxy
>

You built this proxy with --with-default-user=proxy , which sets the 
default value of cache_effective_user to "proxy", no need to repeat that 
in squid.conf.

>     icap_enable on
>     icap_send_client_ip on
>     icap_send_client_username on
>     icap_client_username_encode off
>     icap_client_username_header X-Authenticated-User
>     icap_preview_enable on
>     icap_preview_size 1024
>     icap_service service_req reqmod_precache bypass=0
>     icap://127.0.0.1:1344/squidclamav
>     icap_service service_resp respmod_precache bypass=0
>     icap://127.0.0.1:1344/squidclamav
>     adaptation_access service_req allow all
>     adaptation_access service_resp allow all
>     redirect_program /usr/bin/squidGuard -c
>     /etc/squidguard/squidGuard.conf
>     cache_effective_group proxy
>

There should be no need for that cache_effective_group directive to be 
used. Simply check and limit the groups the cache_effective_user account 
is a member of.

>     dns_nameservers 8.8.8.8
>

Using that DNS service directly in Squid is particularly nasty. Each DNS 
query usually hits a different server in their farm and thus gets 
different set of response IP addresses. I strongly recommend that you 
setup some local DNS recursive resolver that both the clients and Squid 
can use. That resolver can of course pass its traffic to 8.8.8.8 if you 
actually need to.

Amos