[squid-users] filtering http(s) sites, transparently

Tue Apr 5 01:23:28 UTC 2016

On 5/04/2016 4:49 a.m., Jok Thuau wrote:
> On Sun, Apr 3, 2016 at 9:59 PM, Amos Jeffries <squid3 at treenet.co.nz> wrote:
> 
>> On 4/04/2016 4:18 p.m., Jok Thuau wrote:
>>> I'm attempting to build a transparent proxy (policy based routing on
>>> firewall to squid proxy) with the following behavior:
>>>
>>> 1) proxies http traffic for a given set of domains, provide an message
>>> otherwise such "domain not allowed" or similar
>>> 2) proxies https traffic for a given set of domains (ideally, splicing
>>> those, so as not to break HSTS, if enabled), otherwise provide an error
>>> message (bumping and providing "domain not allowed")
>>>
>>> I'm attempting this with a 3.5.15 compiled with icap (not yet used) and
>>> ssl-bumping.
>>>
>>> Part 1 seems easy enough (and is well documented)...
>>>
>>> acl whitelist dstdomain .domain1.tld
>>> acl whitelist dstdomain .domain2.tld
>>>
>>> acl http_ok all-of whitelist !SSL_ports
>>>
>>> http_access allow http_ok
>>> http_access deny all
>>
>> This is denying the HTTPS traffic CONNECT requests (synthesized by
>> Squid), since they only have IP address no domain name.
>>
> 
> yes, this is where I started with just http.
> 
> 
>>
>>
>>>
>>> Moving onto Part 2 (the peek and splice setup) appears to be the topic
>> of a
>>> few discussions out there...
>>>
>>> acl sni_whitelist ssl::server_name .domain1.tld
>>> acl sni_whitelist ssl::server_name .domain2.tld
>>>
>>> ssl_bump peek step1
>>
>> You have omitted the definition of step1 ACL.
>>
>>
> the definition of "step1" is the same as the one on the wiki (See full
> config below)
> 
> 
>>> ssl_bump splice sni_whitelist
>>> ssl_bump bump all
>>>
>>> It appears however that when combining the two, the generated
>>> certificate(s), instead of mimic'ing the original server's certificate
>>> comes out with the CN=<IP> where <IP> is the ip used by the "connect"
>> part
>>> of the connection. In addition, it appears that only the first entry ever
>>> matches (at this point, i've tried so many combinations, i'm no longer
>>> certain of anything).
>>
>> You have omitted the http(s)_port configuration details, and the step1
>> ACL. So its not possible to say if you have the cert generation settings
>> wrong, or if the peeking step is matching wrong, or something else.
>>
>>
> That's included in the config below.
> 
> 
>>>
>>> If i remove *all* the http_access lines, then the behavior appears
>> correct
>>> (from a "splicing/bumping" standpoint).
>>>
>>
>> Strange. Squid without any http_access lines should be denying traffic
>> 100%.
>>
>>
> I do not see this behavior. Traffic appears to be allowed, and bumped
> (though with the wrong certificate, depending on the config, as explained
> before).
> 
> 
>>> Can anyone confirm that this is indeed possible to achieve?
>>>
>>> I believe, based on experimentation that any http_access i have, because
>> of
>>> the "deny all" cause the bumping to "short circuit" and effectively send
>> an
>>> early "access denied" based on the only information it has (the ip
>> address
>>> from the "connect", rather than the SNI that would come later).
>>>
>>> Would a setup where "deny http+!whitelist" so have the allow be the
>> default
>>> allow for the bumping to work and get to step2 and match the sni* acls
>>> somehow? (with a "deny step2 !sni_whitelist").
>>>
>>> Is 3.5.15 capable of doing this? If this requires some feature/effort,
>> what
>>> would be the procedure to sponsor that work?
>>
>> It is not possible to answer any of those questsions properly without
>> full config details. You have omitted a lot.
>>
> 
> my apologies for trying to show only the relevant parts. Find below the
> current config.
> It appears to be bumping everything rather than splicing any of the config
> (which may be due to the limitations documented on the wiki)
> 
> acl Safe_ports port 80 # http
> acl Safe_ports port 443 # https
> acl SSL_ports port 443
> acl CONNECT method CONNECT
> http_port 3129 intercept
> https_port 8443 intercept ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=64MB \
>     cert=/etc/squid/ssl/proxy.pem \
>     key=/etc/squid/ssl/proxy.key \
>     cafile=/etc/squid/ssl/proxy.pem
> always_direct allow all

always_direct has not been necessary with SSL-Bump sice 3.1 series. You
should remove it.

> acl step1 at_step SslBump1
> acl step2 at_step SslBump2
> acl step3 at_step SslBump3
> acl SniBypass ssl::server_name_regex \.slashdot\.org
> acl SniBypass ssl::server_name_regex \.fsdn\.com
> acl http_bypass dstdomain .slashdot.org
> acl http_bypass dstdomain .fsdn.com
> acl https_bypass all-of CONNECT SniBypass

This https_bypass ACL definition is a bit weird. It requires a single
message to match both TLS and HTTP properties simultaneously.

As you might imagine it is difficult for a TLS messages to match HTTP
properties, and vice versa. So it wont ever match.

Note: SNI is *not* equivalent to Host or URL domain name. They can
contain very different values. The only thing they have in common is
that they both are supposed to point at the IP of the server being
contacted.

> acl http_ok all-of http_bypass Safe_ports
> ssl_bump peek step1
> ssl_bump splice SniBypass step2

This splice will work if (and only if) the client sends TLS SNI values
to Squid. It will ignore the server cert details.

For clients which do not send SNI or for all connections where the SNI
does not match your ACL the bump rule below will do client-first bumping
(without the server cert).

> ssl_bump bump all

I suggets you try these ssl_bump rules instead:

 ssl_bump splice SniBypass
 ssl_bump peek step1
 ssl_bump stare step2
 ssl_bump bump all

> sslproxy_cert_sign_hash sha256
> sslproxy_options NO_SSLv2,NO_SSLv3,SINGLE_ECDH_USE
> always_direct allow all

repeated useless line.

> sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB
> sslcrtd_children 8 startup=1 idle=1
> http_access allow http_ok
> http_access allow CONNECT

 ... implicit "deny all" for everything which is not a CONNECT message
or matching http_bypass.

Okay. That sort of matches your policy. Except that you are missing the
security defaults. Those lines are carefully tuned for the specific
behaviour to protect against security attacks:

 http_access deny !Safe_ports
 http_access deny CONNECT !SSL_ports

.. and should be above your custom rules.

> no_cache allow all
> cache deny all

"no_cache" is an old confusing name for "cache" directive. So what you
have there is actually:

 cache allow all
 cache deny all

... pick one.

> shutdown_lifetime 3 seconds
> 

Amos