<div dir="ltr"><div>Sorry, I haven't seen the reply as it's been attached as an attachment and posted a similar question about 302, but probably I know the answer already as it's not stateful inspection any redirection to a new domain is a new request that has to go over ACL again. I thought about using http_reply_access but again it's creating a whole new ACL that every reply would have to go over not just that one. Is there a way to use http_access & http_reply_access together, like http_access allow whitelist & http_reply allow whitelist? <br></div><div>Mirek<br></div><div><br></div><div>> There is a built-in ACL called "all" which does what you defined for the regex "blacklist" to do.<br>> As for sessions. No Squid follows HTTP which is stateless. You can configure it though. setup an ext_session_acl helper for active mode sessions that start when a 302 response comes back. you should have some other ACL to separately whitelist the sites normally blocked, but can open with a session.<br>> Amos</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Apr 15, 2021 at 9:52 PM Miroslaw Malinowski <<a href="mailto:mr.miroslaw.malinowski@gmail.com">mr.miroslaw.malinowski@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>I've found a resolution using a bit better regex: <br></div><div><br></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">acl blackList url_regex ^https?:\/\/.*$<br></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><br></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">looking at the debug it doing exactly what I wanted, however, I now have a different issue how to handle a 302 MOVED when the move is to a different domain, e.g. <span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><a href="http://packages.gitlab.com" target="_blank">packages.gitlab.com</a> are moved to <span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><a href="http://d20rj4el6vkp4c.cloudfront.net" target="_blank">d20rj4el6vkp4c.cloudfront.net</a>. Is squid stateful in a way that it's able to remember those packets are coming from the same session? What would be the best way to resolve the issue other than just keep adding domain if a thing like this happens.</span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><br></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><br></span></span></span></span></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">Thanks</span><br></span></span></span></span></span></div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Apr 15, 2021 at 1:03 PM Miroslaw Malinowski <<a href="mailto:mr.miroslaw.malinowski@gmail.com" target="_blank">mr.miroslaw.malinowski@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi,</div><div><br></div><div>I'm trying to use Opnsense built-in squid config to set up a transparent proxy for server updates and block everything else.</div><div>In GUI they use url_regex for whitelist and blacklist, when I simple per domain whitelist and blacklist it's working as expected, e.g.</div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"># ACL - Whitelist - User defined (whiteList)
</span><br>acl whiteList url_regex archive\.ubuntu\.com<br></span></div><div><span style="font-family:monospace"></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"># ACL - Blacklist - User defined (blackList)
</span><br>acl blackList url_regex packages\.gitlab\.com</span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"># ACL list (Allow) whitelist
</span><br>http_access allow whiteList
<br># ACL list (Deny) blacklist
<br>http_access deny blackList<br></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><br></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace">However, when I do wildcard in blacklist I also get all https domain blocked even when I've tried to explicitly allow it with <a href="https://archive" target="_blank">https://archive</a>\.ubuntu\.com , e.g.</span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"></span></span></span></div><div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"># ACL - Whitelist - User defined (whiteList)
</span><br>acl whiteList url_regex archive\.ubuntu\.com<br></span></div><div><span style="font-family:monospace"></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"># ACL - Blacklist - User defined (blackList)
</span><br>acl blackList url_regex .*</span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"># ACL list (Allow) whitelist
</span><br>http_access allow whiteList
<br># ACL list (Deny) blacklist
<br>http_access deny blackList</span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><br></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace">I get:</span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">Err:7 <a href="https://repos.influxdata.com/ubuntu" target="_blank">https://repos.influxdata.com/ubuntu</a> focal InRelease
</span><br> 403 Forbidden [IP: 52.84.95.46 443]</span></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><br></span></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace">What I'm trying to say is with blacklist as . is blocking all https traffic even if whitelisted, is this an expected behaviour or I'm doing something wrong or it can't be done with url_regex and I should do it at backend manually.</span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><br></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace">My config:</span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">#
</span><br># Automatic generated configuration for Squid.
<br># Do not edit this file manually.
<br>#
<br>
<br>
<br># Setup transparent mode listeners on loopback interfaces
<br>http_port <a href="http://127.0.0.1:3128" target="_blank">127.0.0.1:3128</a> intercept ssl-bump cert=/var/squid/ssl/ca.pem dynamic_cert_mem_cache_size=10MB generate-host-certificates=on
<br>http_port [::1]:3128 intercept ssl-bump cert=/var/squid/ssl/ca.pem dynamic_cert_mem_cache_size=10MB generate-host-certificates=on
<br>https_port <a href="http://127.0.0.1:3129" target="_blank">127.0.0.1:3129</a> intercept ssl-bump cert=/var/squid/ssl/ca.pem dynamic_cert_mem_cache_size=10MB generate-host-certificates=on
<br>https_port [::1]:3129 intercept ssl-bump cert=/var/squid/ssl/ca.pem dynamic_cert_mem_cache_size=10MB generate-host-certificates=on
<br>
<br># Setup regular listeners configuration
<br>http_port <a href="http://172.16.230.252:3128" target="_blank">172.16.230.252:3128</a> ssl-bump cert=/var/squid/ssl/ca.pem dynamic_cert_mem_cache_size=10MB generate-host-certificates=on
<br>http_port <a href="http://172.16.230.254:3128" target="_blank">172.16.230.254:3128</a> ssl-bump cert=/var/squid/ssl/ca.pem dynamic_cert_mem_cache_size=10MB generate-host-certificates=on
<br>
<br># setup ssl re-cert
<br>sslcrtd_program /usr/local/libexec/squid/security_file_certgen -s /var/squid/ssl_crtd -M 10MB
<br>sslcrtd_children 5
<br>
<br>tls_outgoing_options options=NO_TLSv1 cipher=HIGH:MEDIUM:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS
<br>
<br># setup ssl bump acl's
<br>acl bump_step1 at_step SslBump1
<br>acl bump_step2 at_step SslBump2
<br>acl bump_step3 at_step SslBump3
<br>acl bump_nobumpsites ssl::server_name "/usr/local/etc/squid/nobumpsites.acl"
<br>
<br># configure bump
<br>ssl_bump peek bump_step1 all
<br>ssl_bump peek bump_step2 bump_nobumpsites
<br>ssl_bump splice bump_step3 bump_nobumpsites
<br>ssl_bump stare bump_step2
<br>ssl_bump bump bump_step3
<br>
<br>sslproxy_cert_error deny all
<br>
<br>acl ftp proto FTP
<br>http_access allow ftp
<br>
<br>
<br># Setup ftp proxy
<br>
<br># Rules allowing access from your local networks.
<br># Generated list of (internal) IP networks from where browsing
<br># should be allowed. (Allow interface subnets).
<br>acl localnet src <net>/24 # Possible internal network (interfaces v4)
<br># Default allow for local-link and private networks
<br>acl localnet src fc00::/7 # RFC 4193 local private network range
<br>acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
<br>
<br># ACL - Allow localhost for PURGE cache if enabled
<br>acl PURGE method PURGE
<br>http_access allow localhost PURGE
<br>http_access deny PURGE
<br>
<br># ACL lists
<br># ACL - Whitelist - User defined (whiteList)
<br>acl whiteList url_regex packages\.wazuh\.com
<br>acl whiteList url_regex archive\.ubuntu\.com
<br>acl whiteList url_regex security\.ubuntu\.com
<br>acl whiteList url_regex repos\.influxdata\.com
<br>
<br># ACL - Blacklist - User defined (blackList)
<br>acl blackList url_regex .*
<br>
<br># ACL - Remote fetched Blacklist (remoteblacklist)
<br>
<br># ACL - Block browser/user-agent - User defined (browser)
<br>
<br># ACL - SSL ports, default are configured in config.xml
<br># Configured SSL ports (if defaults are not listed, then they have been removed from the configuration!):
<br>acl SSL_ports port 443 # https
<br>
<br># Default Safe ports are now defined in config.xml
<br># Configured Safe ports (if defaults are not listed, then they have been removed from the configuration!):
<br># ACL - Safe_ports
<br>acl Safe_ports port 80 # http
<br>acl Safe_ports port 21 # ftp
<br>acl Safe_ports port 443 # https
<br>acl Safe_ports port 70 # gopher
<br>acl Safe_ports port 210 # wais
<br>acl Safe_ports port 1025-65535 # unregistered ports
<br>acl Safe_ports port 280 # http-mgmt
<br>acl Safe_ports port 488 # gss-http
<br>acl Safe_ports port 591 # filemaker
<br>acl Safe_ports port 777 # multiling http
<br>acl CONNECT method CONNECT
<br>
<br># ICAP SETTINGS
<br># disable icap
<br>icap_enable off
<br>
<br># Pre-auth plugins
<br>include /usr/local/etc/squid/pre-auth/*.conf
<br>
<br># Authentication Settings
<br>
<br># ACL list (Allow) whitelist
<br>http_access allow whiteList
<br>
<br></span></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace">
#
<br># ACL list (Deny) blacklist
<br>http_access deny blackList
<br>
<br># Google Suite Filter
<br>
<br># YouTube Filter
<br>
<br># Deny requests to certain unsafe ports
<br>
<br>http_access deny !Safe_ports <br># Deny CONNECT to other than secure SSL ports
<br>
<br>http_access deny CONNECT !SSL_ports <br>
<br># Only allow cachemgr access from localhost
<br>http_access allow localhost manager
<br>http_access deny manager
<br>
<br># We strongly recommend the following be uncommented to protect innocent
<br># web applications running on the proxy server who think the only
<br># one who can access services on "localhost" is a local user
<br>http_access deny to_localhost
<br>
<br># Auth plugins
<br>include /usr/local/etc/squid/auth/*.conf
<br>
<br>#
<br># Access Permission configuration:
<br>#
<br># Deny request from unauthorized clients
<br>
<br>#
<br># ACL - localnet - default these include ranges from selected interfaces (Allow local subnets)
<br>http_access allow localnet
<br>
<br># ACL - localhost
<br>http_access allow localhost
<br>
<br># Deny all other access to this proxy
<br>http_access deny all
<br># Post-auth plugins
<br>include /usr/local/etc/squid/post-auth/*.conf
<br>
<br># Caching settings
<br>cache_mem 1000 MB
<br>maximum_object_size 200 MB
<br>cache_replacement_policy heap LFUDA
<br>cache_dir ufs /var/squid/cache 100000 16 256
<br>
<br># Leave coredumps in the first cache dir
<br>coredump_dir /var/squid/cache
<br>
<br>#
<br># Add any of your own refresh_pattern entries above these.
<br>#
<br>
<br># Linux package cache:
<br>refresh_pattern pkg\.tar\.xz$ 0 20% 4320 refresh-ims
<br>refresh_pattern d?rpm$ 0 20% 4320 refresh-ims
<br>refresh_pattern deb$ 0 20% 4320 refresh-ims
<br>refresh_pattern udeb$ 0 20% 4320 refresh-ims
<br>refresh_pattern Packages\.bz2$ 0 20% 4320 refresh-ims
<br>refresh_pattern Sources\.bz2$ 0 20% 4320 refresh-ims
<br>refresh_pattern Release\.gpg$ 0 20% 4320 refresh-ims
<br>refresh_pattern Release$ 0 20% 4320 refresh-ims
<br># <a href="http://wiki.squid-cache.org/SquidFaq/WindowsUpdate" target="_blank">http://wiki.squid-cache.org/SquidFaq/WindowsUpdate</a>
<br>refresh_pattern -i <a href="http://microsoft.com/.*%5C.(cab%7Cexe%7Cms%5Bi%7Cu%7Cf%5D%7C%5Bap%5Dsf%7Cwm%5Bv%7Ca%5D%7Cdat%7Czip%7Cesd)" target="_blank">microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip|esd)</a> 4320 80% 129600 reload-into-ims
<br>refresh_pattern -i <a href="http://windowsupdate.com/.*%5C.(cab%7Cexe%7Cms%5Bi%7Cu%7Cf%5D%7C%5Bap%5Dsf%7Cwm%5Bv%7Ca%5D%7Cdat%7Czip%7Cesd)" target="_blank">windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip|esd)</a> 4320 80% 129600 reload-into-ims
<br>refresh_pattern -i <a href="http://windows.com/.*%5C.(cab%7Cexe%7Cms%5Bi%7Cu%7Cf%5D%7C%5Bap%5Dsf%7Cwm%5Bv%7Ca%5D%7Cdat%7Czip%7Cesd)" target="_blank">windows.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip|esd)</a> 4320 80% 129600 reload-into-ims
<br>
<br>refresh_pattern ^ftp: 1440 20% 10080
<br>refresh_pattern ^gopher: 1440 0% 1440
<br>refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
<br>refresh_pattern . 0 20% 4320
<br>
<br># Squid Options
<br># dns_v4_first reverses the order of preference to make Squid contact dual-stack websites over IPv4 first
<br>dns_v4_first on
<br>pinger_enable off
<br>access_log stdio:/var/log/squid/access.log squid
<br>cache_store_log stdio:/var/log/squid/store.log
<br># URI hanlding with Whitespaces (default=strip)
<br>uri_whitespace strip
<br># X-Forwarded header handling (default=on)
<br>forwarded_for on
<br># Disable squid logfile rotate to use system defaults
<br>logfile_rotate 0
<br># Define visible email
<br>cache_mgr admin@localhost.local
<br>error_directory /usr/local/etc/squid/errors/local<br></span></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><br></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace">Thanks<br></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><br></span></span></span></div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"></span></span></span></div></div>
</blockquote></div>
</blockquote></div>