<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Thanks but even with the --no-check-certificate option and using a bump instead of splicing, it still fails as shown above unless I add the localnet rule. The question is: why does the same ACL line:
<div><br>
</div>
<div>http_access allow whitelist</div>
<div><br>
</div>
<div>suddenly work when I add an unrelated ACL line after it (http_access allow localnet)? Why does it correctly determine the domain httpbin.org in the later case as shown by cache.log?</div>
<div><br>
</div>
<div>I updated my splice to a bump instead so my config looks like this:</div>
<div><br>
</div>
<div>acl whitelist ssl::server_name .httpbin.org</div>
<div>acl whitelist_http ssl::server_name .httpbin.org</div>
<div><br>
</div>
<div>acl step1 at_step SslBump1</div>
<div>acl step2 at_step SslBump2</div>
<div>acl step3 at_step SslBump3</div>
<div>ssl_bump peek step1</div>
<div>ssl_bump bump all</div>
<div><br>
</div>
<div>http_access allow whitelist</div>
<div>http_access allow whitelist_http</div>
<div><br>
</div>
<div>#http_access allow localnet</div>
<div>http_access deny all</div>
<div><br>
</div>
<div>and so it will bump all connections:</div>
<div><br>
</div>
<div>I then ran:</div>
<div><br>
</div>
<div>wget https://httpbin.org --no-check-certificate</div>
<div><br>
</div>
<div>and get the following (it fails):</div>
<div><br>
</div>
<div>cat access.log</div>
<div><br>
</div>
<div>1598316641.358 3 192.168.123.214 TCP_DENIED/200 0 CONNECT 3.220.112.94:443 - HIER_NONE/- -</div>
<div>1598316641.366 1 192.168.123.214 NONE/403 3789 GET https://httpbin.org/ - HIER_NONE/- text/html</div>
<div><br>
</div>
<div>cat cache.log | grep -i httpbin.org</div>
<div><br>
</div>
<div>2020/08/24 20:50:41.356 kid1| 28,7| ServerName.cc(32) aclHostDomainCompare: Match:3.220.112.94 <> .httpbin.org</div>
<div>2020/08/24 20:50:41.356 kid1| 28,7| ServerName.cc(32) aclHostDomainCompare: Match:3.220.112.94 <> .httpbin.org</div>
<div><br>
</div>
<div>while once I add the localnet rule:</div>
<div><br>
</div>
<div>http_access allow localnet</div>
<div><br>
</div>
<div>it succeeds and access.log looks like this:</div>
<div><br>
</div>
<div>1598316682.044 753 192.168.123.214 NONE/200 0 CONNECT 54.236.246.173:443 - ORIGINAL_DST/54.236.246.173 -</div>
<div>1598316682.329 260 192.168.123.214 TCP_REFRESH_MODIFIED/200 9936 GET https://httpbin.org/ - ORIGINAL_DST/54.236.246.173 text/html</div>
<div><br>
</div>
<div>and cache.log shows that it actually does the proper domain comparison:</div>
<div><br>
</div>
<div>cat cache.log | grep -i httpbin.org</div>
<div><br>
</div>
<div>2020/08/24 20:51:21.292 kid1| 28,7| ServerName.cc(32) aclHostDomainCompare: Match:54.236.246.173 <> .httpbin.org</div>
<div>2020/08/24 20:51:21.292 kid1| 28,7| ServerName.cc(32) aclHostDomainCompare: Match:54.236.246.173 <> .httpbin.org</div>
<div>2020/08/24 20:51:22.071 kid1| 28,3| RegexData.cc(43) match: checking 'https://httpbin.org/'</div>
<div>2020/08/24 20:51:22.071 kid1| 28,4| ServerName.cc(82) check_cert_domain: Verifying certificate name/subjectAltName httpbin.org</div>
<div>2020/08/24 20:51:22.071 kid1| 28,3| ServerName.cc(42) match: checking 'httpbin.org'</div>
<div>2020/08/24 20:51:22.071 kid1| 28,7| ServerName.cc(32) aclHostDomainCompare: Match:httpbin.org <> .httpbin.org</div>
<div>2020/08/24 20:51:22.071 kid1| 28,3| ServerName.cc(47) match: 'httpbin.org' found</div>
<div><br>
</div>
<div>From my understanding, Squid should perform the exact same steps in both cases BUT then allow the connection because of the localnet ACL line that it sees right before the deny all line, not because it suddenly was able to match httpbin.org using a domain
compare as shown by the debug logs. What am I missing?</div>
<div><br>
</div>
<div>Even if I add a peek at step2, the behavior is still the same for the first scenario although I no longer need to use --no-check-certificate in the 2nd scenario (where I whitelist localnet)</div>
<div><br>
</div>
<div>PS. I'm using the current iptable rules and making the wget call from a normal user account (so neither root or proxy):</div>
<div><br>
</div>
<div>iptables -t nat -A OUTPUT -p tcp -m tcp --dport 80 -m owner --uid-owner root -j RETURN</div>
<div>iptables -t nat -A OUTPUT -p tcp -m tcp --dport 80 -m owner --uid-owner proxy -j RETURN</div>
<div>iptables -t nat -A OUTPUT -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3129</div>
<div>iptables -t nat -A OUTPUT -p tcp -m tcp --dport 443 -m owner --uid-owner root -j RETURN</div>
<div>iptables -t nat -A OUTPUT -p tcp -m tcp --dport 443 -m owner --uid-owner proxy -j RETURN</div>
<div>iptables -t nat -A OUTPUT -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 3130</div>
<br>
<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Alex Rousskov <rousskov@measurement-factory.com><br>
<b>Sent:</b> Tuesday, August 25, 2020 8:41 AM<br>
<b>To:</b> Mathew Brown <mbrown8918@outlook.com>; squid-users@lists.squid-cache.org <squid-users@lists.squid-cache.org><br>
<b>Subject:</b> Re: [squid-users] Strange Squid SSL Interception Behavior</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">On 8/24/20 6:21 PM, Mathew Brown wrote:<br>
<br>
> acl whitelist ssl::server_name .httpbin.org<br>
> acl whitelist_http ssl::server_name .httpbin.org<br>
<br>
> ssl_bump peek step1<br>
> ssl_bump splice all<br>
<br>
> http_access allow whitelist<br>
> http_access allow whitelist_http<br>
> http_access deny all<br>
<br>
The rules above only allow CONNECT requests to .httpbin.org domains.<br>
<br>
During step1, when Squid intercepts a TLS connection to an IP address of<br>
an .httpbin.org domain, Squid http_access rules are applied to a (fake)<br>
CONNECT request to the destination IP address. There are no domain names<br>
at that TCP-level bumping stage. Thus, you place your Squid at the mercy<br>
of reverse DSN lookups.<br>
<br>
In my environment, reverse DNS does not work for httpbin.org the way you<br>
may expect:<br>
<br>
> $ host 54.236.246.173<br>
> 173.246.236.54.in-addr.arpa domain name pointer ec2-54-236-246-173.compute-1.amazonaws.com.<br>
<br>
The above AWS domain name does not match your whitelist ACLs, of course,<br>
and, hence, the fake CONNECT request is denied. Denied requests are<br>
bumped to deliver the error message. Bumped requests require<br>
--no-check-certificate or other means of trusting Squid's CA certificate.<br>
<br>
If you cannot explicitly allow CONNECT requests to httpbin.org IP<br>
addresses (e.g., because they change too often), then consider allowing<br>
CONNECT to safe ports at any address at step1. If you only intercept<br>
connections to httpbin.org IPs, then you can probably relax your step1<br>
http_access rules to allow all CONNECTs (to safe addresses).<br>
<br>
There are many similar questions about allowing CONNECT to IP addresses<br>
on this mailing list. You may be able to find more detailed advice or<br>
instructions by searching for those mailing list threads.<br>
<br>
<br>
HTH,<br>
<br>
Alex.<br>
<br>
<br>
> $ wget <a href="https://httpbin.org">https://httpbin.org</a><br>
> --2020-08-24 17:48:34-- <a href="https://httpbin.org/">https://httpbin.org/</a><br>
> Resolving httpbin.org (httpbin.org)... 54.236.246.173, 3.220.112.94<br>
> Connecting to httpbin.org (httpbin.org)|54.236.246.173|:443... connected.<br>
> ERROR: cannot verify httpbin.org's certificate, issued by ‘O=Internet<br>
> Widgits Pty Ltd,ST=Some-State,C=AU’:<br>
> Self-signed certificate encountered.<br>
> To connect to httpbin.org insecurely, use `--no-check-certificate'.<br>
> <br>
> $ wget <a href="https://httpbin.org">https://httpbin.org</a> --no-check-certificate<br>
> --2020-08-24 17:48:40-- <a href="https://httpbin.org/">https://httpbin.org/</a><br>
> Resolving httpbin.org (httpbin.org)... 3.220.112.94, 54.236.246.173<br>
> Connecting to httpbin.org (httpbin.org)|3.220.112.94|:443... connected.<br>
> WARNING: cannot verify httpbin.org's certificate, issued by ‘O=Internet<br>
> Widgits Pty Ltd,ST=Some-State,C=AU’:<br>
> Self-signed certificate encountered.<br>
> HTTP request sent, awaiting response... 403 Forbidden<br>
> 2020-08-24 17:48:40 ERROR 403: Forbidden.<br>
> <br>
> looking at access.log shows:<br>
> <br>
> 1598305800.974 2 192.168.123.214 TCP_DENIED/200 0 CONNECT<br>
> 54.236.246.173:443 - HIER_NONE/- -<br>
> <br>
> for the first request (without the --no-check-certificate) and the<br>
> following for the 2nd request (with the --no-check-certificate):<br>
> <br>
> 1598305812.292 3 192.168.123.214 TCP_DENIED/200 0 CONNECT<br>
> 54.236.246.173:443 - HIER_NONE/- -<br>
> 1598305812.300 2 192.168.123.214 NONE/403 3795 GET<br>
> <a href="https://httpbin.org/">https://httpbin.org/</a> - HIER_NONE/- text/html<br>
> <br>
> looking at cache.log shows:<br>
> <br>
> # cat /var/log/squid/cache.log | grep -i "28" | grep -i httpbin<br>
> 2020/08/24 17:50:00.972 kid1| 28,7| ServerName.cc(32)<br>
> aclHostDomainCompare: Match:54.236.246.173 <> .httpbin.org<br>
> 2020/08/24 17:50:00.972 kid1| 28,7| ServerName.cc(32)<br>
> aclHostDomainCompare: Match:54.236.246.173 <> .httpbin.org<br>
> 2020/08/24 17:50:12.290 kid1| 28,7| ServerName.cc(32)<br>
> aclHostDomainCompare: Match:54.236.246.173 <> .httpbin.org<br>
> 2020/08/24 17:50:12.290 kid1| 28,7| ServerName.cc(32)<br>
> aclHostDomainCompare: Match:54.236.246.173 <> .httpbin.org<br>
> <br>
> so it never matches on the httpbin.org<br>
> <br>
> now, if I add the following line to my configuration:<br>
> <br>
> http_access allow localnet<br>
> <br>
> right before the:<br>
> <br>
> http_access deny all<br>
> <br>
> line it works and I see the following in access.log:<br>
> <br>
> 1598305979.004 4 192.168.123.214 NONE/200 0 CONNECT<br>
> 54.236.246.173:443 - HIER_NONE/- -<br>
> 1598305980.016 1012 192.168.123.214 TCP_TUNNEL/200 15370 CONNECT<br>
> httpbin.org:443 - ORIGINAL_DST/54.236.246.173 -<br>
> <br>
> and I see the following in cache.log:<br>
> <br>
> # cat /var/log/squid/cache.log | grep -i "28" | grep -i httpbin<br>
> 2020/08/24 17:52:59.000 kid1| 28,7| ServerName.cc(32)<br>
> aclHostDomainCompare: Match:54.236.246.173 <> .httpbin.org<br>
> 2020/08/24 17:52:59.000 kid1| 28,7| ServerName.cc(32)<br>
> aclHostDomainCompare: Match:54.236.246.173 <> .httpbin.org<br>
> 2020/08/24 17:52:59.005 kid1| 28,3| RegexData.cc(43) match: checking<br>
> 'httpbin.org:443'<br>
> 2020/08/24 17:52:59.005 kid1| 28,3| ServerName.cc(42) match: checking<br>
> 'httpbin.org'<br>
> 2020/08/24 17:52:59.005 kid1| 28,7| ServerName.cc(32)<br>
> aclHostDomainCompare: Match:httpbin.org <> .httpbin.org<br>
> 2020/08/24 17:52:59.005 kid1| 28,3| ServerName.cc(47) match:<br>
> 'httpbin.org' found<br>
> <br>
> What's puzzling is why adding the 'allow localnet' line changes the ACL<br>
> logic for .httpbin.org and why the original configuration does not work.<br>
> Any ideas? Thanks<br>
> <br>
> PS. I reproduced the exact same scenario on Ubuntu 20.04 with Squid 4.12<br>
> <br>
> <br>
> <br>
> _______________________________________________<br>
> squid-users mailing list<br>
> squid-users@lists.squid-cache.org<br>
> <a href="http://lists.squid-cache.org/listinfo/squid-users">http://lists.squid-cache.org/listinfo/squid-users</a><br>
> <br>
<br>
</div>
</span></font></div>
</body>
</html>