[squid-users] How to tell HTTPS traffic is using cache from access.log in 3.5.x when using ssl_bump

[Amos Jeffries]

On 26/07/17 10:56, Lei Wen wrote:
> Hi,
> 
> I am setting up a transparent proxy that is doing whitelist work, and at 
> the same time, doing the cache work.
> 
> 
> The whitelist works fine, HTTP cache verifed work cause I see 
> TCP_MEM_HIT using this squid.conf, but don't see any HTTPS MEM HIT 
> related log, every time seems a new connection.


Nod. HIT and REFRESH log entries show cache usage. Squid versions with 
SSL-Bump support are more likely to show REFRESH.


> 
> For HTTPS traffic, I am getting TCP_TUNNEL/200 all the time, the 
> question is, how can I tell that a HTTPS traffic is actually using 
> cache, or in this case, it's not using cache at all for HTTPS. I am 
> using forward-proxy port in cache_peer.

That TUNNEL shows SSL-Bump splice action happening, or SSL-Bump not even 
being considered (ie traffic received in your Squids' port 3130 and 3128).


> 
> I understand that there is logformat to make access.log show hostname 
> instead of ip, but this should not effect the MEM HIT log, right?
> 
> Meanwhile, I am also trying to setup the sibling cache cluster, not sure 
> if this related to HTTPS cache, I am also getting TCP_DENIED/403 
> for squid-internal-dynamic/netdb - HIER_NONE/- text/html. I do see 
> sibling hit for HTTP site.
> 

You have configured splice or terminate. No bumping / decryption will 
ever happen, so the cache is never even considered for use.


> And my squid.conf
> 
> # Squid normally listens to port 3128
> 
> http_port 3130
> 
> 
> http_port 3128 intercept
> 
> acl allowed_http_sites dstdomain "/etc/squid3/whitelist.txt"
> 
> http_access allow allowed_http_sites
> 
> 
> https_port 3129 cert=/etc/squid3/squid.crt key=/etc/squid3/squid.key 
> ssl-bump intercept generate-host-certificates=on 
> dynamic_cert_mem_cache_size=4MB
> 
> acl SSL_port port 443
> 
> http_access allow SSL_port
> 

There are now almost no restrictions on port 443. Anyone can get through 
it so long as they claim to be contacting one of the allowed_https_sites 
(a the HTTP CONNECT level, not the TLS is unrestricted).
That includes traffic in the ports 3130 and 3128 which are not 
considered for ssl_bump processing.



> acl allowed_https_sites ssl::server_name "/etc/squid3/ssl_sites.txt"
> 
> #sslcrtd_program /usr/local/squid/libexec/ssl_crtd -s /var/lib/ssl_db -M 4MB
> 
> sslcrtd_program /lib/squid3/ssl_crtd -s /var/lib/ssl_db -M 4MB
> 
> 
> 
> acl step1 at_step SslBump1
> 
> acl step2 at_step SslBump2
> 
> acl step3 at_step SslBump3
> 
> ssl_bump peek step1 all
> 
> ssl_bump peek step2 allowed_https_sites

dstdomain ACL is not reliable in ssl_bump. It uses the HTTP URI domain 
from the previous CONNECT instead of the actual TLS details.
Use ssl::server_name ACL type instead.

> 
> ssl_bump splice step3 allowed_https_sites
> 
> ssl_bump terminate step2 all

So what happens when the server TLS cert reveals it is not actually 
serving up one of the allowed_https_sites? nothing, the traffic is 
allowed through.


Note the terminate line only gets used at step2 (ie. based on client SNI 
claims alone). The peek at step2 has precluded / prohibited bump from 
happening at step3, which only leaves splice as being possible.


Amos