[squid-users] Strange delays (30 seconds) with TLS connections in WCCP/Transparent mode

[Christophe Fillot]

Hello all,

I have a strange problem where some TLS connections are delayed by 30 
seconds when going through my transparent proxy with WCCP. This occurs 
typically with sites behind Cloudflare (for example, 
https://www.wireshark.org). No problem for Google websites for example.

I only want to log the SNI hostname, I do *not* want to 
intercept/decrypt/re-encrypt connections with fake certificates.

Here is the setup:

     - Linux Debian 7
     - OpenSSL 1.0.1t
     - Squid 3.5.23 (also tested Squid 4.0.17).

The Squid configure options are:

./configure --enable-delay-pools --with-large-files --enable-async-io 
--enable-icmp --enable-kill-parent-hack
--enable-htpc --enable-forw-via-db --enable-cache-digests 
--enable-dl-malloc --with-large-files
--enable-linux-netfilter --enable-ssl --enable-ssl-crtd --with-openssl

IPTables configuration (the routing device sending WCCP frames is a 
Cisco ASA):

iptables -t nat -A PREROUTING -i wccp0 -p tcp -m tcp --dport 443 -j DNAT 
--to-destination 195.83.155.53:3130

The output/public interface is eth0, the traffic returns to clients 
through eth1 with the following:

iptables -t mangle -A OUTPUT -p tcp --sport 3130  -j MARK --set-mark 900
ip rule add fwmark 900 table 1

(The table 1 allows direct access to client networks with the 
appropriate routes, this is needed because the return traffic must not 
go through the ASA).

The SQUID configuration:

https_port 3130 intercept ssl-bump cert=/usr/local/squid/etc/proxyCA.pem

acl step1 at_step SslBump1
acl step2 at_step SslBump2
ssl_bump peek step1 all
ssl_bump splice step2 all

sslcrtd_program  /usr/local/squid/libexec/ssl_crtd -s 
/usr/local/squid/var/ssl_db -M 40MB
sslcrtd_children 5
wccp2_service dynamic 70 password=XXXXXXXXX
wccp2_service_info 70 protocol=tcp flags=src_ip_hash,dst_ip_hash 
priority=240 ports=443

Even if I use only "ssl_bump splice all" the 30-second delay happens.

I have an example of captured traffic here, on the client and the 
various network interfaces on the proxy server: 
http://www.utc.fr/filex/get?k=6Dt169xGsHMswCKEF5L

As you can see in the captures, Squid returns the "Server Hello" 30 
seconds (in cap_eth1.cap) after it has received it from the server (in 
cap_eth0.cap).

This behavior is not systematic, sometimes the data is returned 
immediately. What could cause this delay ? This looks like some timeout, 
but for what reason ?

Thanks in advance for any suggestion !

Christophe