[squid-users] Bridge/Tproxy: https dns

Tue Aug 18 05:30:01 UTC 2015

On 18/08/2015 12:25 a.m., Pedro Correia Sardinha wrote:
> Hello,
> 
> I'm trying to setup a Squid server simple as possible just to review the
> web use in office using the last stable version 3.5.7.
> 

And you chose TPROXY with ssl-bump'ing. The two most complex features to
setup. lol.

> I setup the bridge with 2 NIC, br0 with IP 192.168.0.5 and I had disable
> IPv6 on boot in my Slackware Current (Fri Aug 14 2015) server.

Sigh. Ever heard of IPv6-over-IPv4, 6-in-4, 6to4, etc. ?
All protocols designed to "fix" connectivity going through machines
setup like yours.

And why bother disabling a (BCP 177) mandatory part of the kernel?
The correct way to handle unwanted traffic is to firewall it. Not to
play around with kernel internals.

<snip>
> My squid.conf:

> tcp_outgoing_address 85.138.204.43

This is irrelevant with TPROXY. The client IP address is used instead.

For the regular forward-proxy traffic on port 3128 the machines default
IP will be used.

> dns_v4_first on
> pinger_enable off
> http_port 3128
> http_port 3129 tproxy
> https_port 3130 ssl-bump tproxy generate-host-certificates=off
> cert=/etc/squid/ssl/squid.pem cafile=/etc/squid/ssl/squid.pem
> always_direct allow ssl-bump_port
> ssl_bump none all

You have configued Squid not to even look at the TLS details.

> dns_nameservers 8.8.8.8 8.8.4.4
> access_log daemon:/var/log/squid/access.log squid
> cache deny all
> pid_filename /var/run/squid/squid.pid
> coredump_dir /var/log/squid/cache/squid
> visible_hostname myservername.domain.local
> 
> In general the configuration (squid.conf) it's working but has some
> incomplete behaviors as shows in log files.
> 
> access.log (I know this is Facebook but there are no dns resolusion in
> https, just IP):
> 1439811492.625   2377 192.168.0.102 TCP_TUNNEL/200 3574 CONNECT
> 31.13.90.2:443 - ORIGINAL_DST/31.13.90.2 -

What sort of resolution were you expecting?

* The above log line is recording the TCP connection. TCP packets do not
have any "domain name" fields that need resolving to IP addresses.

* you also configured Squid not to look at the TLS details where it
might have found an SNI entry with server domain name.

The result is that Squid is working purely with TPROXY IP addresses and
setting up a TCP tunnel to relay the traffic through.

> 
> cache.log:
> HTTP/1.1 200 OK
> Accept-Ranges: bytes
> Cache-Control: max-age=504747
> Content-Type: application/ocsp-response
> Date: Mon, 17 Aug 2015 11:38:03 GMT
> ETag: "55d15943-1d7"
> Expires: Sun, 23 Aug 2015 23:38:03 GMT
> Last-Modified: Mon, 17 Aug 2015 03:47:15 GMT
> Server: ECS (mad/439C)
> X-Cache: HIT
> Content-Length: 471
> X-Cache: MISS from squidhead2.skywalker.local
> Via: 1.1 squidhead2.skywalker.local (squid/3.5.7)
> Connection: keep-alive
> ----------
> 2015/08/17 12:38:09.067 kid1| store.cc(955) checkCachable:
> StoreEntry::checkCachable: NO: not cachable
> 2015/08/17 12:38:09.067 kid1| store.cc(955) checkCachable:
> StoreEntry::checkCachable: NO: not cachable
> 2015/08/17 12:38:09.067 kid1| store.cc(955) checkCachable:
> StoreEntry::checkCachable: NO: not cachable

Above is all the end of some transaction that was started earlier. No
useful details in the provided log snippet about it.

This is where a transaction actually starts:

> 2015/08/17 12:38:10.248 kid1| TcpAcceptor.cc(222) doAccept: New connection
> on FD 12
> 2015/08/17 12:38:10.248 kid1| TcpAcceptor.cc(297) acceptNext: connection on
> local=0.0.0.0:3130 remote=[::] FD 12 flags=25
> 2015/08/17 12:38:10.248 kid1| client_side.cc(3890)

Notice how the connection local IP:port details change from port 3130 to
port 443. Thats TPROXY working.

> httpsSslBumpAccessCheckDone: sslBump not needed for local=31.13.90.2:443
> remote=192.168.0.102 FD 50 flags=17

This is the "ssl_bump none" action working (by not doing anything TLS
related) exactly as you configured.

Squid is now processing an internally generated CONNECT request
representing the intercepted TPROXY connection in a way that can be
logged and/or relayed to other proxies if it needs to.

> 2015/08/17 12:38:10.248 kid1| client_side.cc(2337) parseHttpRequest: HTTP
> Client local=31.13.90.2:443 remote=192.168.0.102 FD 50 flags=17
> 2015/08/17 12:38:10.248 kid1| client_side.cc(2338) parseHttpRequest: HTTP
> Client REQUEST:
> ---------
> CONNECT 31.13.90.2:443 HTTP/1.1
> Host: 31.13.90.2:443
> ---------
> 2015/08/17 12:38:10.248 kid1| client_side_request.cc(741)
> clientAccessCheckDone: The request CONNECT 31.13.90.2:443 is ALLOWED; last
> ACL checked: localnet
> 2015/08/17 12:38:10.248 kid1| client_side_request.cc(717)
> clientAccessCheck2: No adapted_http_access configuration. default: ALLOW
> 2015/08/17 12:38:10.248 kid1| client_side_request.cc(741)
> clientAccessCheckDone: The request CONNECT 31.13.90.2:443 is ALLOWED; last
> ACL checked: localnet
> 2015/08/17 12:38:10.248 kid1| peer_select.cc(280) peerSelectDnsPaths: Found
> sources for '31.13.90.2:443'

The CONNECT request uses a raw-IP address provided by TPROXY. There is
no name to resolve.

> 2015/08/17 12:38:10.248 kid1| peer_select.cc(281) peerSelectDnsPaths:
> always_direct = ALLOWED
> 2015/08/17 12:38:10.248 kid1| peer_select.cc(282) peerSelectDnsPaths:
>  never_direct = DENIED
> 2015/08/17 12:38:10.248 kid1| peer_select.cc(288) peerSelectDnsPaths:
>  ORIGINAL_DST = local=192.168.0.102 remote=31.13.90.2:443 flags=25
> 2015/08/17 12:38:10.248 kid1| peer_select.cc(295) peerSelectDnsPaths:
>  timedout = 0

... stuff happens for 2 seconds...

> 2015/08/17 12:38:12.621 kid1| client_side.cc(815) swanSong: local=
> 31.13.90.2:443 remote=192.168.0.102 flags=17
> 2015/08/17 12:38:12.625 kid1| client_side.cc(815) swanSong: local=
> 31.13.90.2:443 remote=192.168.0.102 flags=17

Then the connection closes.

Looks perfectly normal and expected behaviour to me considering what you
configured.

> 
> 
> The logs with http (port 80) has the name resolution of navigation.
> 
> I disabled pinger because give some error:
<snip>
> 2015/08/17 12:49:55| FATAL: pinger: Unable to open any ICMP sockets.

Okay. Not a big problem. The pinger helper needs its suid bit set, which
is not working on all system installations yet. Disabling it is fine.

> Do I have to setup local DNS server? the internal DNS of squid can't handle
> https in Tproxy?
> What's missing to have name resolution in https traffic as its showed in
> http traffic?

Whats missing is the SSL-bumping part. HTTPS works differently to HTTP.

The URL domain name and all the rest of the HTTP message is encrypted
*in full*. There is simply no client HTTP message involved if you don't
decrypt.

As I mentioned above what you are seeing in the log is a Squid-generated
CONNECT message. Its the HTTP representation of the intercepted TCP SYN
packet and contains purely raw-IP:port details.

TLS does have an SNI record which is sent by browsers un-encrypted that
can be used as domain for some things. BUT, that requires "ssl_bump
peek" action at minimum, and has no guarantee of actually being present.

SNI is also still a new feature, and is not used for these fake CONNECT
requests anyway (since they represent the TCP SYN). So you wont see any
log difference in 3.5 even if you do let your Squid use it for ACLs.

Amos