[squid-users] block user agent
Yuri
yvoinov at gmail.com
Fri Nov 17 15:29:54 UTC 2017
17.11.2017 21:27, Vieri пишет:
> ________________________________
> From: Alex Rousskov <rousskov at measurement-factory.com>
>> 1. Your "works" and "does not work" setups currently differ in at least
>> three variables: user agent name, slash after the user agent name, and
>> acl negation in http_access. Find out which single variable is
>> responsible for the breakage by eliminating all other differences.
>>
>> 2. Post two ALL,2 cache.logs, each containing a single transaction, one
>> for the "works" case and one for the "does not work" case polished as
>> discussed in #1.
>
>
> I can't really do anything about #1 except maybe leave out the forward slash.
> That's because my 2 examples are trying to achieve the opposite.
> Let me just rephrase everything so it's crystal clear.
>
> My goal is to deny all client traffic from browsers that DO NOT have a specific user-agent string. So this is a negated statement. One of the things I can't do in Squid is define an ACL with a negated lookahead such as (?!useragentname).
I hope you listen about browser extensions for UA spoofing?
>
> So I set up two examples.
>
> Common to both:
>
> acl allowed_useragent browser MyAllowedUAstring
> acl denied_useragent browser MyDeniedUAstring
>
> # example 1:
> http_access deny denied_useragent
> http_reply_access deny denied_useragent
> deny_info http://proxy-server1/proxy-error/?a=%a&B=%B&e=%e&E=%E&H=%H&i=%i&M=%M&o=%o&R=%R&T=%T&U=%U&u=%u&w=%w&x=%x&acl=denied_useragent denied_useragent
>
> I then run this from my test client:
>
> # curl --insecure --user-agent MyAllowedUAstring https://www.gentoo.org
> -> works as expected (I see the web site). I guess you don't need to see cache.log here.
>
> Now I run this:
>
> # curl --insecure --user-agent MyDeniedUAstring https://www.gentoo.org
> -> works as expected (I'm denied access and I see Squid's error page).
> I guess there's no need for the full log here either. It boils down to this anyway:
> 2017/11/17 13:24:26.937 kid1| 28,2| RegexData.cc(73) match: aclRegexData::match: match '(MyDeniedUAstring)' found in 'MyDeniedUAstring'
> 2017/11/17 13:24:26.937 kid1| 85,2| client_side_request.cc(745) clientAccessCheckDone: The request GET https://www.gentoo.org/ is DENIED; last ACL checked: denied_useragent
>
> I'm done with example 1. That's because I cannot make a consistent list of all user agents I want to actively block. Instead, I want to "deny everyone except one or two".
>
> Also, since negative lookaheads are not supported in regular expressions, I change my example 1 to:
>
> # example 2:
> http_access deny !allowed_useragent
> http_reply_access deny !allowed_useragent
> deny_info http://proxy-server1/proxy-error/?a=%a&B=%B&e=%e&E=%E&H=%H&i=%i&M=%M&o=%o&R=%R&T=%T&U=%U&u=%u&w=%w&x=%x&acl=denied_useragent allowed_useragent
>
> Then I run this from the client:
>
> # curl --insecure --user-agent MyAllowedUAstring https://www.gentoo.org
> -> I was expecting to be allowed access since Squid denies "everything that's not" MyAllowedUAstring. Well, at least I should have passed the "deny" line in example 2.
> However, I'm being blocked right there. This is the full log:
>
> 2017/11/17 13:30:42.216 kid1| 5,2| TcpAcceptor.cc(220) doAccept: New connection on FD 88
> 2017/11/17 13:30:42.216 kid1| 5,2| TcpAcceptor.cc(295) acceptNext: connection on local=[::]:3229 remote=[::] FD 88 flags=25
> 2017/11/17 13:30:42.216 kid1| 33,2| client_side.cc(3943) httpsSslBumpAccessCheckDone: sslBump needed for local=89.16.167.134:443 remote=10.215.144.48 FD 8 flags=17 method 4
> 2017/11/17 13:30:42.216 kid1| 11,2| client_side.cc(2372) parseHttpRequest: HTTP Client local=89.16.167.134:443 remote=10.215.144.48 FD 8 flags=17
> 2017/11/17 13:30:42.216 kid1| 11,2| client_side.cc(2373) parseHttpRequest: HTTP Client REQUEST:
> ---------
> CONNECT 89.16.167.134:443 HTTP/1.1
> Host: 89.16.167.134:443
>
>
> ----------
> 2017/11/17 13:30:42.216 kid1| 85,2| client_side_request.cc(745) clientAccessCheckDone: The request CONNECT 89.16.167.134:443 is DENIED; last ACL checked: allowed_useragent
> 2017/11/17 13:30:42.216 kid1| 20,2| store.cc(996) checkCachable: StoreEntry::checkCachable: NO: not cachable
> 2017/11/17 13:30:42.216 kid1| 20,2| store.cc(996) checkCachable: StoreEntry::checkCachable: NO: not cachable
> 2017/11/17 13:30:42.216 kid1| 20,2| store.cc(996) checkCachable: StoreEntry::checkCachable: NO: not cachable
> 2017/11/17 13:30:42.226 kid1| 83,2| client_side.cc(3843) clientNegotiateSSL: clientNegotiateSSL: New session 0x125e030 on FD 8 (10.215.144.48:65262)
> 2017/11/17 13:30:42.226 kid1| 11,2| client_side.cc(2372) parseHttpRequest: HTTP Client local=89.16.167.134:443 remote=10.215.144.48 FD 8 flags=17
> 2017/11/17 13:30:42.226 kid1| 11,2| client_side.cc(2373) parseHttpRequest: HTTP Client REQUEST:
> ---------
> GET / HTTP/1.1
> Host: www.gentoo.org
> User-Agent: MyAllowedUAstring
> Accept: */*
>
>
> ----------
> 2017/11/17 13:30:42.227 kid1| 28,2| RegexData.cc(73) match: aclRegexData::match: match '(MyAllowedUAstring)' found in 'MyAllowedUAstring'
> 2017/11/17 13:30:42.227 kid1| 88,2| client_side_reply.cc(2073) processReplyAccessResult: The reply for GET https://www.gentoo.org/ is ALLOWED, because it matched denied_mimetypes_rep
> 2017/11/17 13:30:42.227 kid1| 11,2| client_side.cc(1409) sendStartOfMessage: HTTP Client local=89.16.167.134:443 remote=10.215.144.48 FD 8 flags=17
> 2017/11/17 13:30:42.227 kid1| 11,2| client_side.cc(1410) sendStartOfMessage: HTTP Client REPLY:
> ---------
> HTTP/1.1 307 Temporary Redirect
> Server: squid
> Mime-Version: 1.0
> Date: Fri, 17 Nov 2017 12:30:42 GMT
> Content-Type: text/html;charset=utf-8
> Content-Length: 0
> Location: http://proxy-server1/proxy-error/?a=-&B=&e=0&E=%5BNo%20Error%5D&H=89.16.167.134&i=10.215.144.48&M=CONNECT&o=&R=/&T=Fri,%2017%20Nov%202017%2012%3A30%3A42%20GMT&U=https%3A%2F%2F89.16.167.134%2F*&u=89.16.167.134%3A443&w=IT%40mydomain.org&x=&acl=denied_useragent
> X-Squid-Error: 403 Access Denied
> X-Cache: MISS from proxy-server1
> X-Cache-Lookup: NONE from proxy-server1:3227
> Connection: close
>
> Note that I have these defaults in my squid conf file:
>
> acl CONNECT method CONNECT
> http_access deny CONNECT !SSL_ports
>
> Let's try another one:
>
> # curl --insecure --user-agent MyDeniedUAstring https://www.gentoo.org
> -> This is as expected, I guess.
>
> Full log:
>
> 2017/11/17 13:30:10.365 kid1| 5,2| TcpAcceptor.cc(220) doAccept: New connection on FD 88
> 2017/11/17 13:30:10.365 kid1| 5,2| TcpAcceptor.cc(295) acceptNext: connection on local=[::]:3229 remote=[::] FD 88 flags=25
> 2017/11/17 13:30:10.365 kid1| 33,2| client_side.cc(3943) httpsSslBumpAccessCheckDone: sslBump needed for local=89.16.167.134:443 remote=10.215.144.48 FD 8 flags=17 method 4
> 2017/11/17 13:30:10.365 kid1| 11,2| client_side.cc(2372) parseHttpRequest: HTTP Client local=89.16.167.134:443 remote=10.215.144.48 FD 8 flags=17
> 2017/11/17 13:30:10.365 kid1| 11,2| client_side.cc(2373) parseHttpRequest: HTTP Client REQUEST:
> ---------
> CONNECT 89.16.167.134:443 HTTP/1.1
> Host: 89.16.167.134:443
>
>
> ----------
> 2017/11/17 13:30:10.365 kid1| 85,2| client_side_request.cc(745) clientAccessCheckDone: The request CONNECT 89.16.167.134:443 is DENIED; last ACL checked: allowed_useragent
> 2017/11/17 13:30:10.365 kid1| 20,2| store.cc(996) checkCachable: StoreEntry::checkCachable: NO: not cachable
> 2017/11/17 13:30:10.365 kid1| 20,2| store.cc(996) checkCachable: StoreEntry::checkCachable: NO: not cachable
> 2017/11/17 13:30:10.365 kid1| 20,2| store.cc(996) checkCachable: StoreEntry::checkCachable: NO: not cachable
> 2017/11/17 13:30:10.385 kid1| 83,2| client_side.cc(3843) clientNegotiateSSL: clientNegotiateSSL: New session 0xdbdc70 on FD 8 (10.215.144.48:65237)
> 2017/11/17 13:30:10.386 kid1| 11,2| client_side.cc(2372) parseHttpRequest: HTTP Client local=89.16.167.134:443 remote=10.215.144.48 FD 8 flags=17
> 2017/11/17 13:30:10.386 kid1| 11,2| client_side.cc(2373) parseHttpRequest: HTTP Client REQUEST:
> ---------
> GET / HTTP/1.1
> Host: www.gentoo.org
> User-Agent: MyDeniedUAstring
> Accept: */*
>
>
> ----------
> 2017/11/17 13:30:10.386 kid1| 88,2| client_side_reply.cc(2073) processReplyAccessResult: The reply for GET https://www.gentoo.org/ is DENIED, because it matched allowed_useragent
> 2017/11/17 13:30:10.386 kid1| 20,2| store.cc(996) checkCachable: StoreEntry::checkCachable: NO: not cachable
> 2017/11/17 13:30:10.386 kid1| 20,2| store.cc(996) checkCachable: StoreEntry::checkCachable: NO: not cachable
> 2017/11/17 13:30:10.386 kid1| 20,2| store.cc(996) checkCachable: StoreEntry::checkCachable: NO: not cachable
> 2017/11/17 13:30:10.386 kid1| 88,2| client_side_reply.cc(2073) processReplyAccessResult: The reply for GET https://www.gentoo.org/ is ALLOWED, because it matched allowed_useragent
> 2017/11/17 13:30:10.386 kid1| 11,2| client_side.cc(1409) sendStartOfMessage: HTTP Client local=89.16.167.134:443 remote=10.215.144.48 FD 8 flags=17
> 2017/11/17 13:30:10.386 kid1| 11,2| client_side.cc(1410) sendStartOfMessage: HTTP Client REPLY:
> ---------
> HTTP/1.1 302 Found
> Server: squid
> Mime-Version: 1.0
> Date: Fri, 17 Nov 2017 12:30:10 GMT
> Content-Type: text/html;charset=utf-8
> Content-Length: 0
> Location: http://proxy-server1/proxy-error/?a=-&B=&e=0&E=%5BNo%20Error%5D&H=www.gentoo.org&i=10.215.144.48&M=GET&o=&R=/&T=Fri,%2017%20Nov%202017%2012%3A30%3A10%20GMT&U=https%3A%2F%2Fwww.gentoo.org%2F&u=https%3A%2F%2Fwww.gentoo.org%2F&w=IT%40mydomain.org&x=&acl=denied_useragent
> X-Squid-Error: 403 Access Denied
> X-Cache: MISS from proxy-server1
> X-Cache-Lookup: NONE from proxy-server1:3227
> Connection: close
>
> Now for plain HTTP with example 2.
>
> # curl --user-agent MyDeniedUAstring http://www.fltk.org/index.php
> -> As expected. It blocks access.
>
> Full log:
>
> 2017/11/17 15:56:52.648 kid1| 5,2| TcpAcceptor.cc(220) doAccept: New connection on FD 85
> 2017/11/17 15:56:52.648 kid1| 5,2| TcpAcceptor.cc(295) acceptNext: connection on local=[::]:3228 remote=[::] FD 85 flags=25
> 2017/11/17 15:56:52.648 kid1| 11,2| client_side.cc(2372) parseHttpRequest: HTTP Client local=66.39.46.122:80 remote=10.215.144.48 FD 8 flags=17
> 2017/11/17 15:56:52.648 kid1| 11,2| client_side.cc(2373) parseHttpRequest: HTTP Client REQUEST:
> ---------
> GET /index.php HTTP/1.1
> Host: www.fltk.org
> User-Agent: MyDeniedUAstring
> Accept: */*
>
>
> ----------
> 2017/11/17 15:56:52.648 kid1| 85,2| client_side_request.cc(745) clientAccessCheckDone: The request GET http://www.fltk.org/index.php is DENIED; last ACL checked: allowed_useragent
> 2017/11/17 15:56:52.648 kid1| 20,2| store.cc(996) checkCachable: StoreEntry::checkCachable: NO: not cachable
> 2017/11/17 15:56:52.648 kid1| 20,2| store.cc(996) checkCachable: StoreEntry::checkCachable: NO: not cachable
> 2017/11/17 15:56:52.648 kid1| 20,2| store.cc(996) checkCachable: StoreEntry::checkCachable: NO: not cachable
> 2017/11/17 15:56:52.648 kid1| 88,2| client_side_reply.cc(2073) processReplyAccessResult: The reply for GET http://www.fltk.org/index.php is ALLOWED, because it matched allowed_useragent
> 2017/11/17 15:56:52.648 kid1| 11,2| client_side.cc(1409) sendStartOfMessage: HTTP Client local=66.39.46.122:80 remote=10.215.144.48 FD 8 flags=17
> 2017/11/17 15:56:52.648 kid1| 11,2| client_side.cc(1410) sendStartOfMessage: HTTP Client REPLY:
> ---------
> HTTP/1.1 302 Found
> Server: squid
> Mime-Version: 1.0
> Date: Fri, 17 Nov 2017 14:56:52 GMT
> Content-Type: text/html;charset=utf-8
> Content-Length: 0
> Location: http://proxy-server1/proxy-error/?a=-&B=&e=0&E=%5BNo%20Error%5D&H=www.fltk.org&i=10.215.144.48&M=GET&o=&R=/index.php&T=Fri,%2017%20Nov%202017%2014%3A56%3A52%20GMT&U=http%3A%2F%2Fwww.fltk.org%2Findex.php&u=http%3A%2F%2Fwww.fltk.org%2Findex.php&w=IT%40mydomain.org&x=&acl=denied_useragent
> X-Squid-Error: 403 Access Denied
> X-Cache: MISS from proxy-server1
> X-Cache-Lookup: NONE from proxy-server1:3227
> Connection: keep-alive
>
> However, now comes the interesting part.
>
> # curl --user-agent MyAllowedUAstring http://www.fltk.org/index.php
> -> works as expected (I see the web site).
>
> Full log:
>
> 2017/11/17 15:55:23.550 kid1| 5,2| TcpAcceptor.cc(220) doAccept: New connection on FD 85
> 2017/11/17 15:55:23.550 kid1| 5,2| TcpAcceptor.cc(295) acceptNext: connection on local=[::]:3228 remote=[::] FD 85 flags=25
> 2017/11/17 15:55:23.551 kid1| 11,2| client_side.cc(2372) parseHttpRequest: HTTP Client local=66.39.46.122:80 remote=10.215.144.48 FD 8 flags=17
> 2017/11/17 15:55:23.551 kid1| 11,2| client_side.cc(2373) parseHttpRequest: HTTP Client REQUEST:
> ---------
> GET /index.php HTTP/1.1
> Host: www.fltk.org
> User-Agent: MyAllowedUAstring
> Accept: */*
>
>
> ----------
> 2017/11/17 15:55:23.551 kid1| 28,2| RegexData.cc(73) match: aclRegexData::match: match '(MyAllowedUAstring)' found in 'MyAllowedUAstring'
> 2017/11/17 15:55:23.551 kid1| 82,2| external_acl.cc(805) aclMatchExternal: bllookup("http www.fltk.org 80 /index.php") = lookup needed
> 2017/11/17 15:55:23.551 kid1| 82,2| external_acl.cc(808) aclMatchExternal: "http www.fltk.org 80 /index.php": queueing a call.
> 2017/11/17 15:55:23.551 kid1| 82,2| external_acl.cc(1444) Start: fg lookup in 'bllookup' for 'http www.fltk.org 80 /index.php'
> 2017/11/17 15:55:23.551 kid1| 82,2| external_acl.cc(811) aclMatchExternal: "http www.fltk.org 80 /index.php": return -1.
> 2017/11/17 15:55:23.553 kid1| 82,2| external_acl.cc(1372) externalAclHandleReply: reply={result=OK, notes={message: www.fltk.org site not found in blacklist; }}
> 2017/11/17 15:55:23.553 kid1| 82,2| external_acl.cc(1288) external_acl_cache_add: external_acl_cache_add: Adding 'http www.fltk.org 80 /index.php' = ALLOWED
> 2017/11/17 15:55:23.553 kid1| 82,2| external_acl.cc(841) aclMatchExternal: bllookup = ALLOWED
> 2017/11/17 15:55:23.553 kid1| 85,2| client_side_request.cc(745) clientAccessCheckDone: The request GET http://www.fltk.org/index.php is ALLOWED; last ACL checked: bl_lookup
> 2017/11/17 15:55:23.553 kid1| 85,2| client_side_request.cc(721) clientAccessCheck2: No adapted_http_access configuration. default: ALLOW
> 2017/11/17 15:55:23.553 kid1| 85,2| client_side_request.cc(745) clientAccessCheckDone: The request GET http://www.fltk.org/index.php is ALLOWED; last ACL checked: bl_lookup
> 2017/11/17 15:55:23.554 kid1| 88,2| client_side_reply.cc(593) cacheHit: clientProcessHit: Vary detected!
> 2017/11/17 15:55:23.554 kid1| 17,2| FwdState.cc(133) FwdState: Forwarding client request local=66.39.46.122:80 remote=10.215.144.48 FD 8 flags=17, url=http://www.fltk.org/index.php
> 2017/11/17 15:55:23.554 kid1| 44,2| peer_select.cc(280) peerSelectDnsPaths: Found sources for 'http://www.fltk.org/index.php'
> 2017/11/17 15:55:23.554 kid1| 44,2| peer_select.cc(281) peerSelectDnsPaths: always_direct = DENIED
> 2017/11/17 15:55:23.554 kid1| 44,2| peer_select.cc(282) peerSelectDnsPaths: never_direct = DENIED
> 2017/11/17 15:55:23.554 kid1| 44,2| peer_select.cc(288) peerSelectDnsPaths: ORIGINAL_DST = local=10.215.144.48 remote=66.39.46.122:80 flags=25
> 2017/11/17 15:55:23.554 kid1| 44,2| peer_select.cc(295) peerSelectDnsPaths: timedout = 0
> 2017/11/17 15:55:23.708 kid1| 11,2| http.cc(2229) sendRequest: HTTP Server local=10.215.144.48:35373 remote=66.39.46.122:80 FD 13 flags=25
> 2017/11/17 15:55:23.708 kid1| 11,2| http.cc(2230) sendRequest: HTTP Server REQUEST:
> ---------
> GET /index.php HTTP/1.1
> User-Agent: MyAllowedUAstring
> Accept: */*
> Host: www.fltk.org
> Cache-Control: max-age=259200
> Connection: keep-alive
>
>
> ----------
> 2017/11/17 15:55:23.884 kid1| ctx: enter level 0: 'http://www.fltk.org/index.php'
> 2017/11/17 15:55:23.884 kid1| 11,2| http.cc(719) processReplyHeader: HTTP Server local=10.215.144.48:35373 remote=66.39.46.122:80 FD 13 flags=25
> 2017/11/17 15:55:23.884 kid1| 11,2| http.cc(720) processReplyHeader: HTTP Server REPLY:
> ---------
> HTTP/1.1 200 OK
> Date: Fri, 17 Nov 2017 14:55:23 GMT
> Server: Apache/2.4.29
> Cache-Control: no-cache
> Vary: Accept-Encoding
> Keep-Alive: timeout=5, max=100
> Connection: Keep-Alive
> Transfer-Encoding: chunked
> Content-Type: text/html
>
> 3a02
> <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
> <html>
> <head>
> <title>Fast Light Toolkit - Fast Light Toolkit (FLTK)</title>
> <meta http-equiv='Pragma' content='no-cache'>
> <meta http-equiv='Content-Type' content='text/html; charset=utf-8'>
> <link rel='stylesheet' type='text/css' href='fltk.css'>
> <link rel='alternate' title='FLTK RSS' type='application/rss+xml' href='index.rss'>
> <link rel='shortcut icon' href='favicon.ico' type='image/x-icon'>
> <meta name='keywords' content='gui toolkit,c++,linux,unix,macos x,x11,windows'>
> </head>
> <body>
> <table width='100%' border='0' cellspacing='0' cellpadding='0' summary='Page'>
> <tr class='header'><td valign='top' width='15' rowspan='2'><a href='index.php'><img src='images/top-left.gif' width='15' height='70' border='0' alt=''></a></td><td valign='top' width='224' rowspan='2'><a href='index.php'><img src='images/top-middle.gif' width='224' height='70' border='0' alt=''></a></td><td width='100%' height='40'><h1>Fast Light Toolkit</h1> </td><td align='right' nowrap>
> <table cellpadding=0 cellspacing=0 border=0><tr><td valign=top nowrap>
> <a href=fltk-rss.xml><img src=images/rss-fee
> ----------
> 2017/11/17 15:55:23.885 kid1| ctx: exit level 0
> 2017/11/17 15:55:23.885 kid1| 23,2| url.cc(407) urlParse: urlParse: URI has whitespace: {icap://127.0.0.1:1344/clamav ICAP/1.0
> }
> 2017/11/17 15:55:24.038 kid1| 28,2| RegexData.cc(73) match: aclRegexData::match: match '(MyAllowedUAstring)' found in 'MyAllowedUAstring'
> 2017/11/17 15:55:24.038 kid1| 88,2| client_side_reply.cc(2073) processReplyAccessResult: The reply for GET http://www.fltk.org/index.php is ALLOWED, because it matched denied_mimetypes_rep
> 2017/11/17 15:55:24.038 kid1| 11,2| client_side.cc(1409) sendStartOfMessage: HTTP Client local=66.39.46.122:80 remote=10.215.144.48 FD 8 flags=17
> 2017/11/17 15:55:24.038 kid1| 11,2| client_side.cc(1410) sendStartOfMessage: HTTP Client REPLY:
> ---------
> HTTP/1.1 200 OK
> Date: Fri, 17 Nov 2017 14:55:23 GMT
> Server: Apache/2.4.29
> Cache-Control: no-cache
> Vary: Accept-Encoding
> Content-Type: text/html
> Via: ICAP/1.0 proxy-server1.hospitalmanacor.org (C-ICAP/0.5.2 SquidClamav/Antivirus service )
> X-Cache: MISS from proxy-server1
> X-Cache-Lookup: MISS from proxy-server1:3227
> Transfer-Encoding: chunked
> Connection: keep-alive
>
> How can I modify my example 2 settings so this access control works the same way with both http and https in an ssl-bumped environment.
>
> Thanks,
>
> Vieri
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
--
**************************
* C++: Bug to the future *
**************************
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 648 bytes
Desc: OpenPGP digital signature
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20171117/e436875c/attachment-0001.sig>
More information about the squid-users
mailing list