[squid-users] Web Whatsapp, Dropbox... problem

[Eliezer Croitoru]

As a partial solution until I will be able to sit on the dumps and get the required data I wrote this script:
https://gist.github.com/elico/e0faadf0cc63942c5aaade808a87deef

Which bypasses squid for specific domains.
It is a very simple script and it works OK for whatsapp and it's on the iptables level so your will maybe loss some caching.
But for most of the public critical systems it is much better to have a bypass in standby compared to make your workers work harder then they can.

Eliezer

----
Eliezer Croitoru
Linux System Administrator
Mobile+WhatsApp: +972-5-28704261
Email: eliezer at ngtech.co.il


-----Original Message-----
From: squid-users [mailto:squid-users-bounces at lists.squid-cache.org] On Behalf Of Amos Jeffries
Sent: Thursday, September 29, 2016 7:16 AM
To: squid-users at lists.squid-cache.org
Subject: Re: [squid-users] Web Whatsapp, Dropbox... problem

On 29/09/2016 11:27 a.m., Eliezer Croitoru wrote:
> I am also testing this issue and I have the next settings:
> acl DiscoverSNIHost at_step SslBump1
> acl NoSSLIntercept ssl::server_name_regex -i "/etc/squid/url.nobump"
> ssl_bump splice NoSSLIntercept
> ssl_bump peek DiscoverSNIHost
> ssl_bump bump all
> sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/squid/ssl -M 4MB
> sslcrtd_children 10
> read_ahead_gap 64 MB
> sslproxy_cert_error allow all
> tls_outgoing_options flags=DONT_VERIFY_PEER
> acl foreignProtocol squid_error ERR_PROTOCOL_UNKNOWN ERR_TOO_BIG
> on_unsupported_protocol tunnel foreignProtocol
> 
> (Which is not recommended for production as is!!!)
> 
> Now the "/etc/squid/url.nobump" file contains:
> # WU (Squid 3.5.x and above with SSL Bump)
> # Only this sites must be spliced.
> update\.microsoft\.com$
> update\.microsoft\.com\.akadns\.net$
> v10\.vortex\-win\.data\.microsoft.com$
> settings\-win\.data\.microsoft\.com$
> # The next are trusted SKYPE addresses
> a\.config\.skype\.com$
> pipe\.skype\.com$
> mail\.rimon\.net\.il$
> w[0-9]+\.web\.whatsapp\.com$
> \.web\.whatsapp\.com$
> web\.whatsapp\.com$
> ##END OF NO BUMP DOMAINS.
> 
> And squid 4.0.14 doesn't tunnel the requests.
> The above is with:
> http_port 3128
> http_port 13128 intercept
> https_port 13129 intercept ssl-bump \
>    cert=/etc/squid/ssl_cert/myCA.pem \
>      generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
> 
> On the 443 intercept port.
> Access log output:
> 1475100891.636 000445 192.168.10.112 NONE/200 0 CONNECT 158.85.224.178:443 - ORIGINAL_DST/158.85.224.178 - 52:54:00:bc:9f:73
> 1475100908.469 000223 192.168.10.112 TCP_MISS/200 508 GET https://web.whatsapp.com/status.json - ORIGINAL_DST/31.13.90.51 text/json 52:54:00:bc:9f:73
> 1475100952.107 000445 192.168.10.112 NONE/200 0 CONNECT 158.85.224.178:443 - ORIGINAL_DST/158.85.224.178 - 52:54:00:bc:9f:73
> 1475100968.832 000191 192.168.10.112 NONE/200 0 CONNECT 216.58.214.110:443 - ORIGINAL_DST/216.58.214.110 - 52:54:00:bc:9f:73
> 1475100968.984 000199 192.168.10.112 NONE/200 0 CONNECT 172.217.22.14:443 - ORIGINAL_DST/172.217.22.14 - 52:54:00:bc:9f:73
> 1475101012.572 000447 192.168.10.112 NONE/200 0 CONNECT 158.85.224.178:443 - ORIGINAL_DST/158.85.224.178 - 52:54:00:bc:9f:73
> 1475101033.232 000621 192.168.10.112 NONE/200 0 CONNECT 31.13.66.49:443 - ORIGINAL_DST/31.13.66.49 - 52:54:00:bc:9f:73
> 1475101034.470 001224 192.168.10.112 TCP_MISS/200 512 GET https://web.whatsapp.com/status.json - ORIGINAL_DST/31.13.66.49 text/json 52:54:00:bc:9f:73
> 1475101073.039 000446 192.168.10.112 NONE/200 0 CONNECT 158.85.224.178:443 - ORIGINAL_DST/158.85.224.178 - 52:54:00:bc:9f:73
> 1475101133.502 000448 192.168.10.112 NONE/200 0 CONNECT 158.85.224.178:443 - ORIGINAL_DST/158.85.224.178 - 52:54:00:bc:9f:73
> 
> Now the issue is more then just this since I cannot see any logs about the websocket connections ie to the domains:
> w3.web.whatsapp.com
> 

They might be in the ones with raw-IP in NONE/200 lines. Since
server_name_regex matches against the TLS-cert details which do not
necessarily get logged as a URL domain name when splice is done.

The SNI _should_ be made the CONNECT URI domain. But when it matches the
server cert altSubjectName that is definitely not a client requested value.


> and couple other similar.
> 
> What I did until now is to bypass specific domains IP addresses using ipset+iptables.
> I believe that squid can do much better then it's doing now.

Can you get a packet dump to see what its TLS handshake details actually
are? both client and server sides of Squid.

Amos

_______________________________________________
squid-users mailing list
squid-users at lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users