[squid-users] How to tell HTTPS traffic is using cache from access.log in 3.5.x when using ssl_bump

[Lei Wen]

Hi Amos,

Thanks a lot.
It is my splice thing is blocking proxy in the middle, after using stare
instead of peek, seems work though, terminal in this case is not blocking
proxy in the middle?

I made some change on my squid.conf, it work for http/https caching and
http/https whitelist.
It is working for http sibling cache as well, but has some issue with
https/ssl sibling cache.
in cache_peer, which port number should I use as forward-proxy port?
3130/3128/3129?

I am trying to set up a sibling cache pool, there is no parent or gateway
sort of thing in this hierarchy.
Each instance can have their own whitelist, but they share the same cache
pool.
On each instance host, squid is actually doing it's whitelist and caching
job for a container group on the same host.


http_port 3130

http_port 3128 intercept
acl allowed_http_sites dstdomain "/etc/squid3/whitelist.txt"
http_access allow allowed_http_sites

https_port 3129 cert=/etc/squid3/squid.crt key=/etc/squid3/squid.key
ssl-bump intercept generate-host-certificates=on
dynamic_cert_mem_cache_size=4MB
acl SSL_port port 443
http_access allow SSL_port
acl allowed_https_sites ssl::server_name "/etc/squid3/ssl_sites.txt"

http_access deny all

acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3
ssl_bump stare step1 all
ssl_bump stare step2 allowed_https_sites
#ssl_bump bump step3
ssl_bump terminate step2 all

icp_port 3131
icp_access allow all
cache_peer 10.0.8.48 sibling 3130 3131 proxy-only



Thanks,
Lei

On Tue, Jul 25, 2017 at 9:42 PM, Amos Jeffries <squid3 at treenet.co.nz> wrote:

> On 26/07/17 10:56, Lei Wen wrote:
>
>> Hi,
>>
>> I am setting up a transparent proxy that is doing whitelist work, and at
>> the same time, doing the cache work.
>>
>>
>> The whitelist works fine, HTTP cache verifed work cause I see TCP_MEM_HIT
>> using this squid.conf, but don't see any HTTPS MEM HIT related log, every
>> time seems a new connection.
>>
>
>
> Nod. HIT and REFRESH log entries show cache usage. Squid versions with
> SSL-Bump support are more likely to show REFRESH.
>
>
>
>> For HTTPS traffic, I am getting TCP_TUNNEL/200 all the time, the question
>> is, how can I tell that a HTTPS traffic is actually using cache, or in this
>> case, it's not using cache at all for HTTPS. I am using forward-proxy port
>> in cache_peer.
>>
>
> That TUNNEL shows SSL-Bump splice action happening, or SSL-Bump not even
> being considered (ie traffic received in your Squids' port 3130 and 3128).
>
>
>
>> I understand that there is logformat to make access.log show hostname
>> instead of ip, but this should not effect the MEM HIT log, right?
>>
>> Meanwhile, I am also trying to setup the sibling cache cluster, not sure
>> if this related to HTTPS cache, I am also getting TCP_DENIED/403 for
>> squid-internal-dynamic/netdb - HIER_NONE/- text/html. I do see sibling hit
>> for HTTP site.
>>
>>
> You have configured splice or terminate. No bumping / decryption will ever
> happen, so the cache is never even considered for use.
>
>
> And my squid.conf
>>
>> # Squid normally listens to port 3128
>>
>> http_port 3130
>>
>>
>> http_port 3128 intercept
>>
>> acl allowed_http_sites dstdomain "/etc/squid3/whitelist.txt"
>>
>> http_access allow allowed_http_sites
>>
>>
>> https_port 3129 cert=/etc/squid3/squid.crt key=/etc/squid3/squid.key
>> ssl-bump intercept generate-host-certificates=on
>> dynamic_cert_mem_cache_size=4MB
>>
>> acl SSL_port port 443
>>
>> http_access allow SSL_port
>>
>>
> There are now almost no restrictions on port 443. Anyone can get through
> it so long as they claim to be contacting one of the allowed_https_sites (a
> the HTTP CONNECT level, not the TLS is unrestricted).
> That includes traffic in the ports 3130 and 3128 which are not considered
> for ssl_bump processing.
>
>
>
> acl allowed_https_sites ssl::server_name "/etc/squid3/ssl_sites.txt"
>>
>> #sslcrtd_program /usr/local/squid/libexec/ssl_crtd -s /var/lib/ssl_db -M
>> 4MB
>>
>> sslcrtd_program /lib/squid3/ssl_crtd -s /var/lib/ssl_db -M 4MB
>>
>>
>>
>> acl step1 at_step SslBump1
>>
>> acl step2 at_step SslBump2
>>
>> acl step3 at_step SslBump3
>>
>> ssl_bump peek step1 all
>>
>> ssl_bump peek step2 allowed_https_sites
>>
>
> dstdomain ACL is not reliable in ssl_bump. It uses the HTTP URI domain
> from the previous CONNECT instead of the actual TLS details.
> Use ssl::server_name ACL type instead.
>
>
>> ssl_bump splice step3 allowed_https_sites
>>
>> ssl_bump terminate step2 all
>>
>
> So what happens when the server TLS cert reveals it is not actually
> serving up one of the allowed_https_sites? nothing, the traffic is allowed
> through.
>
>
> Note the terminate line only gets used at step2 (ie. based on client SNI
> claims alone). The peek at step2 has precluded / prohibited bump from
> happening at step3, which only leaves splice as being possible.
>
>
> Amos
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20170726/291a47dd/attachment.html>