[squid-users] Cache Peer Issue with URL

Alex Rousskov rousskov at measurement-factory.com
Sat Jun 5 15:08:24 UTC 2021 

On 6/5/21 11:03 AM, koshik moshik wrote:

> Due to encrypted traffic, is it even possible to get the path for https
> sites?

No, Squid cannot see URL paths of encrypted requests. In an increasing
number of cases, even the exact domain name is not known. Use access.log
to see what URLs Squid sees.

Alex.


> My current squid.conf looks like this and it is not working: 
> 
> http_port 2115
> http_port 2116
> dns_v4_first on
> acl SSL_ports port 1-65535
> acl Safe_ports port 1-65535
> acl CONNECT method CONNECT
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> acl ban_domains dstdomain "/etc/squid/blacklist.txt"
> http_access deny ban_domains
> auth_param basic program /usr/lib/squid/basic_ncsa_auth /etc/squid/.htpasswd
> auth_param basic children 5
> auth_param basic realm Squid Basic Authentication
> auth_param basic credentialsttl 5 hours
> external_acl_type file_userip ipv4 %MYADDR %LOGIN
> /usr/lib/squid/ext_file_userip_acl -f /etc/squid/ip.txt
> external_acl_type file_user %MYPORT %LOGIN
> /usr/lib/squid/ext_file_userip_acl -f /etc/squid/users.txt
> acl password proxy_auth REQUIRED
> acl IP_USER external file_userip
> acl USERS external file_user
> http_access deny !IP_USER
> http_access deny !USERS
> http_access allow password
> http_access deny all
> cache deny all
> never_direct allow all
> 
> 
> 
> 
> acl ab1 urlpath_regex .
> cache_peer my.proxy parent 3128 0 login=user:pw no-query name=user1
> cache_peer_access user1 allow ab1
> cache_peer_access user1 deny all
> 
> 
> 
> 
> acl ab proxy_auth test
> cache_peer my.proxy parent 3128 0 login=user:pw no-query name=user2
> cache_peer_access user2 deny ab1
> cache_peer_access user2 allow all
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> # Leave coredumps in the first cache dir
> coredump_dir /var/spool/squid
> 
> #Rules to anonymize http headers
> forwarded_for off
> request_header_access Allow allow all
> request_header_access Authorization allow all
> request_header_access WWW-Authenticate allow all
> request_header_access Proxy-Authorization allow all
> request_header_access Proxy-Authenticate allow all
> request_header_access Cache-Control allow all
> request_header_access Content-Encoding allow all
> request_header_access Content-Length allow all
> request_header_access Content-Type allow all
> request_header_access Date allow all
> request_header_access Expires allow all
> request_header_access Host allow all
> request_header_access If-Modified-Since allow all
> request_header_access Last-Modified allow all
> request_header_access Location allow all
> request_header_access Pragma allow all
> request_header_access Accept allow all
> request_header_access Accept-Charset allow all
> request_header_access Accept-Encoding allow all
> request_header_access Accept-Language allow all
> request_header_access Content-Language allow all
> request_header_access Mime-Version allow all
> request_header_access Retry-After allow all
> request_header_access Title allow all
> request_header_access Connection allow all
> request_header_access Proxy-Connection allow all
> request_header_access User-Agent allow all
> request_header_access Cookie allow all
> request_header_access All deny all
> 
> #
> # Add any of your own refresh_pattern entries above these.
> #
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
> refresh_pattern (Release|Packages(.gz)*)$      0       20%     2880
> refresh_pattern . 0 20% 4320
> 
> 
> 
> On Sat, Jun 5, 2021 at 1:10 AM Alex Rousskov
> <rousskov at measurement-factory.com
> <mailto:rousskov at measurement-factory.com>> wrote:
> 
>     On 6/4/21 4:19 PM, koshik moshik wrote:
>     > Basically I have created two cashe peers:
>     >
>     > never_direct allow all
>     > acl ab1 dstdom_regex "/etc/squid/Bad_Homepages.squid" cache_peer
>     > my.proxy.com <http://my.proxy.com> <http://my.proxy.com
>     <http://my.proxy.com>> parent 31112 0 login=user:pw no-query
>     > name=user cache_peer_access user allow ab1 cache_peer_access user deny
>     > all acl ab2 proxy_auth userName1 cache_peer myProxyparent 31112 0
>     > login=user:pwno-query name=user2 cache_peer_access user2 allow ab2
>     > cache_peer_access user2 deny all
>     >
>     >
>     > As you can see, I have a Bad_homepages.squid file which contains
>     regular
>     > expressions for a website, so this cache_peer will only be used,
>     if the
>     > HOMEPAGE is requested. If a sub page is requested, the second
>     > cache_peer(acl ab2) should be used. 
>     >
>     > Example: www.test.com <http://www.test.com> <http://www.test.com
>     <http://www.test.com>> -> ab1 should be used 
>     > www.test.com/hello <http://www.test.com/hello>
>     <http://www.test.com/hello <http://www.test.com/hello>> -> ab2
>     should be used. 
> 
>     Sorry about bad quoting -- Thunderbird is having trouble with your HTML
>     email...
> 
> 
>     > The regex in the Bad_Homepages.squid file is following:
>     > .whatismyip.com <http://whatismyip.com>($|/$|/?)
> 
>     It looks like you are using a domain-based dstdom_regex but trying to
>     match URL paths. If you are trying to detect URLs with non-empty paths,
>     then you may want to use something like
> 
>       acl ab1 urlpath_regex .
> 
>     or
> 
>       acl ab1 urlpath_regex /.
> 
>     I do not know whether the leading slash is included and the built-in
>     docs do not say. Others on the list may know the answer or you can find
>     it using tests.
> 
>     You can make the regex tighter by excluding repeated "/" characters and
>     "#", but those are usually unimportant details if you keep in mind that
>     nothing will work reliably in general because the URL path may contain a
>     lot of stuff that the origin server may ignore or reinterpret.
> 
> 
>     Finally, if you want Squid to use either peer A or peer B, then do not
>     use two different ACLs to direct traffic to them. Use mutually exclusive
>     tests of one ACL:
> 
>       cache_peer_access user allow ab1
>       cache_peer_access user deny all
> 
>       cache_peer_access user2 deny ab1
>       cache_peer_access user2 allow all
> 
> 
>     Please be mindful of nonhierarchical_direct and encrypted traffic.
> 
> 
>     HTH,
> 
>     Alex.
> 
> 
>     _______________________________________________
>     squid-users mailing list
>     squid-users at lists.squid-cache.org
>     <mailto:squid-users at lists.squid-cache.org>
>     http://lists.squid-cache.org/listinfo/squid-users
>     <http://lists.squid-cache.org/listinfo/squid-users>
>

More information about the squid-users mailing list