[squid-users] block user agent

Fri Nov 17 16:30:39 UTC 2017

On 18/11/17 04:27, Vieri wrote:
> ________________________________
> From: Alex Rousskov <rousskov at measurement-factory.com>
>> 1. Your "works" and "does not work" setups currently differ in at least
>> three variables: user agent name, slash after the user agent name, and
>> acl negation in http_access. Find out which single variable is
>> responsible for the breakage by eliminating all other differences.
>>
>> 2. Post two ALL,2 cache.logs, each containing a single transaction, one
>> for the "works" case and one for the "does not work" case polished as
>> discussed in #1.
> 
> 
> 
> I can't really do anything about #1 except maybe leave out the forward slash.
> That's because my 2 examples are trying to achieve the opposite.
> Let me just rephrase everything so it's crystal clear.
> 
> My goal is to deny all client traffic from browsers that DO NOT have a specific user-agent string. So this is a negated statement. One of the things I can't do in Squid is define an ACL with a negated lookahead such as (?!useragentname).
> 
> So I set up two examples.
> 
> Common to both:
> 
> acl allowed_useragent browser MyAllowedUAstring
> acl denied_useragent browser MyDeniedUAstring
> 
> # example 1:
> http_access deny denied_useragent
> http_reply_access deny denied_useragent
> deny_info http://proxy-server1/proxy-error/?a=%a&B=%B&e=%e&E=%E&H=%H&i=%i&M=%M&o=%o&R=%R&T=%T&U=%U&u=%u&w=%w&x=%x&acl=denied_useragent denied_useragent
> 
> I then run this from my test client:
> 
> # curl --insecure --user-agent MyAllowedUAstring https://www.gentoo.org
> -> works as expected (I see the web site). I guess you don't need to see cache.log here.
> 
> Now I run this:
> 
> # curl --insecure --user-agent MyDeniedUAstring https://www.gentoo.org
> -> works as expected (I'm denied access and I see Squid's error page).
> I guess there's no need for the full log here either. It boils down to this anyway:
> 2017/11/17 13:24:26.937 kid1| 28,2| RegexData.cc(73) match: aclRegexData::match: match '(MyDeniedUAstring)' found in 'MyDeniedUAstring'
> 2017/11/17 13:24:26.937 kid1| 85,2| client_side_request.cc(745) clientAccessCheckDone: The request GET https://www.gentoo.org/ is DENIED; last ACL checked: denied_useragent
> 
> I'm done with example 1. That's because I cannot make a consistent list of all user agents I want to actively block. Instead, I want to "deny everyone except one or two".
> 
> Also, since negative lookaheads are not supported in regular expressions, I change my example 1 to:
> 
> # example 2:
> http_access deny !allowed_useragent
> http_reply_access deny !allowed_useragent
> deny_info http://proxy-server1/proxy-error/?a=%a&B=%B&e=%e&E=%E&H=%H&i=%i&M=%M&o=%o&R=%R&T=%T&U=%U&u=%u&w=%w&x=%x&acl=denied_useragent allowed_useragent
> 
> Then I run this from the client:
> 
> # curl --insecure --user-agent MyAllowedUAstring https://www.gentoo.org
> -> I was expecting to be allowed access since Squid denies "everything that's not" MyAllowedUAstring. Well, at least I should have passed the "deny" line in example 2.
> However, I'm being blocked right there. This is the full log:
> 
> 2017/11/17 13:30:42.216 kid1| 5,2| TcpAcceptor.cc(220) doAccept: New connection on FD 88
> 2017/11/17 13:30:42.216 kid1| 5,2| TcpAcceptor.cc(295) acceptNext: connection on local=[::]:3229 remote=[::] FD 88 flags=25
> 2017/11/17 13:30:42.216 kid1| 33,2| client_side.cc(3943) httpsSslBumpAccessCheckDone: sslBump needed for local=89.16.167.134:443 remote=10.215.144.48 FD 8 flags=17 method 4
> 2017/11/17 13:30:42.216 kid1| 11,2| client_side.cc(2372) parseHttpRequest: HTTP Client local=89.16.167.134:443 remote=10.215.144.48 FD 8 flags=17
> 2017/11/17 13:30:42.216 kid1| 11,2| client_side.cc(2373) parseHttpRequest: HTTP Client REQUEST:
> ---------
> CONNECT 89.16.167.134:443 HTTP/1.1
> Host: 89.16.167.134:443
> 

This is the CONNECT request generated internally by Squid for the 
bumping process.

> 
> ----------
> 2017/11/17 13:30:42.216 kid1| 85,2| client_side_request.cc(745) clientAccessCheckDone: The request CONNECT 89.16.167.134:443 is DENIED; last ACL checked: allowed_useragent
> 2017/11/17 13:30:42.216 kid1| 20,2| store.cc(996) checkCachable: StoreEntry::checkCachable: NO: not cachable
> 2017/11/17 13:30:42.216 kid1| 20,2| store.cc(996) checkCachable: StoreEntry::checkCachable: NO: not cachable
> 2017/11/17 13:30:42.216 kid1| 20,2| store.cc(996) checkCachable: StoreEntry::checkCachable: NO: not cachable
> 2017/11/17 13:30:42.226 kid1| 83,2| client_side.cc(3843) clientNegotiateSSL: clientNegotiateSSL: New session 0x125e030 on FD 8 (10.215.144.48:65262)
> 2017/11/17 13:30:42.226 kid1| 11,2| client_side.cc(2372) parseHttpRequest: HTTP Client local=89.16.167.134:443 remote=10.215.144.48 FD 8 flags=17
> 2017/11/17 13:30:42.226 kid1| 11,2| client_side.cc(2373) parseHttpRequest: HTTP Client REQUEST:
> ---------
> GET / HTTP/1.1
> Host: www.gentoo.org
> User-Agent: MyAllowedUAstring
> Accept: */*
> 
> 
> ----------
> 2017/11/17 13:30:42.227 kid1| 28,2| RegexData.cc(73) match: aclRegexData::match: match '(MyAllowedUAstring)' found in 'MyAllowedUAstring'
> 2017/11/17 13:30:42.227 kid1| 88,2| client_side_reply.cc(2073) processReplyAccessResult: The reply for GET https://www.gentoo.org/ is ALLOWED, because it matched denied_mimetypes_rep

Please notice the above text and what ACL it is talking about.

Hint: it is NOT the one you are talking about testing.

> 2017/11/17 13:30:42.227 kid1| 11,2| client_side.cc(1409) sendStartOfMessage: HTTP Client local=89.16.167.134:443 remote=10.215.144.48 FD 8 flags=17
> 2017/11/17 13:30:42.227 kid1| 11,2| client_side.cc(1410) sendStartOfMessage: HTTP Client REPLY:
> ---------
> HTTP/1.1 307 Temporary Redirect
> Server: squid
> Mime-Version: 1.0
> Date: Fri, 17 Nov 2017 12:30:42 GMT
> Content-Type: text/html;charset=utf-8
> Content-Length: 0
> Location: http://proxy-server1/proxy-error/?a=-&B=&e=0&E=%5BNo%20Error%5D&H=89.16.167.134&i=10.215.144.48&M=CONNECT&o=&R=/&T=Fri,%2017%20Nov%202017%2012%3A30%3A42%20GMT&U=https%3A%2F%2F89.16.167.134%2F*&u=89.16.167.134%3A443&w=IT%40mydomain.org&x=&acl=denied_useragent
> X-Squid-Error: 403 Access Denied
> X-Cache: MISS from proxy-server1
> X-Cache-Lookup: NONE from proxy-server1:3227
> Connection: close

This is the denial "error" response generated by Squid.

...
> 
> How can I modify my example 2 settings so this access control works the same way with both http and https in an ssl-bumped environment.

It already does. The environment is what is different.

You are looking at Squid generated messages and trying to get them 
replaced with other Squid generated messages simply because they are 
generated by Squid not some arbitrary UA.

If you could replace that Squid generated message with another Squid 
generated message, and replace that Squid generated message with another 
Squid generated message, and replace that Squid generated message with 
another Squid generated message, and replace that Squid generated 
message with another Squid generated message, .... until the machine 
crashes or client gives up waiting and closes the connection.

Amos