<div dir="ltr"><div>Hi,</div><div><br></div><div>I'm trying to use Opnsense built-in squid config to set up a transparent proxy for server updates and block everything else.</div><div>In GUI they use url_regex for whitelist and blacklist, when I simple per domain whitelist and blacklist it's working as expected, e.g.</div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"># ACL - Whitelist - User defined (whiteList)
</span><br>acl whiteList url_regex archive\.ubuntu\.com<br></span></div><div><span style="font-family:monospace"></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"># ACL - Blacklist - User defined (blackList)
</span><br>acl blackList url_regex packages\.gitlab\.com</span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"># ACL list (Allow) whitelist
</span><br>http_access allow whiteList
<br># ACL list (Deny) blacklist
<br>http_access deny blackList<br></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><br></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace">However, when I do wildcard in blacklist I also get all https domain blocked even when I've tried to explicitly allow it with <a href="https://archive">https://archive</a>\.ubuntu\.com , e.g.</span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"></span></span></span></div><div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"># ACL - Whitelist - User defined (whiteList)
</span><br>acl whiteList url_regex archive\.ubuntu\.com<br></span></div><div><span style="font-family:monospace"></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"># ACL - Blacklist - User defined (blackList)
</span><br>acl blackList url_regex .*</span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"># ACL list (Allow) whitelist
</span><br>http_access allow whiteList
<br># ACL list (Deny) blacklist
<br>http_access deny blackList</span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><br></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace">I get:</span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">Err:7 <a href="https://repos.influxdata.com/ubuntu">https://repos.influxdata.com/ubuntu</a> focal InRelease
</span><br> 403 Forbidden [IP: 52.84.95.46 443]</span></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><br></span></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace">What I'm trying to say is with blacklist as . is blocking all https traffic even if whitelisted, is this an expected behaviour or I'm doing something wrong or it can't be done with url_regex and I should do it at backend manually.</span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><br></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace">My config:</span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">#
</span><br># Automatic generated configuration for Squid.
<br># Do not edit this file manually.
<br>#
<br>
<br>
<br># Setup transparent mode listeners on loopback interfaces
<br>http_port <a href="http://127.0.0.1:3128">127.0.0.1:3128</a> intercept ssl-bump cert=/var/squid/ssl/ca.pem dynamic_cert_mem_cache_size=10MB generate-host-certificates=on
<br>http_port [::1]:3128 intercept ssl-bump cert=/var/squid/ssl/ca.pem dynamic_cert_mem_cache_size=10MB generate-host-certificates=on
<br>https_port <a href="http://127.0.0.1:3129">127.0.0.1:3129</a> intercept ssl-bump cert=/var/squid/ssl/ca.pem dynamic_cert_mem_cache_size=10MB generate-host-certificates=on
<br>https_port [::1]:3129 intercept ssl-bump cert=/var/squid/ssl/ca.pem dynamic_cert_mem_cache_size=10MB generate-host-certificates=on
<br>
<br># Setup regular listeners configuration
<br>http_port <a href="http://172.16.230.252:3128">172.16.230.252:3128</a> ssl-bump cert=/var/squid/ssl/ca.pem dynamic_cert_mem_cache_size=10MB generate-host-certificates=on
<br>http_port <a href="http://172.16.230.254:3128">172.16.230.254:3128</a> ssl-bump cert=/var/squid/ssl/ca.pem dynamic_cert_mem_cache_size=10MB generate-host-certificates=on
<br>
<br># setup ssl re-cert
<br>sslcrtd_program /usr/local/libexec/squid/security_file_certgen -s /var/squid/ssl_crtd -M 10MB
<br>sslcrtd_children 5
<br>
<br>tls_outgoing_options options=NO_TLSv1 cipher=HIGH:MEDIUM:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS
<br>
<br># setup ssl bump acl's
<br>acl bump_step1 at_step SslBump1
<br>acl bump_step2 at_step SslBump2
<br>acl bump_step3 at_step SslBump3
<br>acl bump_nobumpsites ssl::server_name "/usr/local/etc/squid/nobumpsites.acl"
<br>
<br># configure bump
<br>ssl_bump peek bump_step1 all
<br>ssl_bump peek bump_step2 bump_nobumpsites
<br>ssl_bump splice bump_step3 bump_nobumpsites
<br>ssl_bump stare bump_step2
<br>ssl_bump bump bump_step3
<br>
<br>sslproxy_cert_error deny all
<br>
<br>acl ftp proto FTP
<br>http_access allow ftp
<br>
<br>
<br># Setup ftp proxy
<br>
<br># Rules allowing access from your local networks.
<br># Generated list of (internal) IP networks from where browsing
<br># should be allowed. (Allow interface subnets).
<br>acl localnet src <net>/24 # Possible internal network (interfaces v4)
<br># Default allow for local-link and private networks
<br>acl localnet src fc00::/7 # RFC 4193 local private network range
<br>acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
<br>
<br># ACL - Allow localhost for PURGE cache if enabled
<br>acl PURGE method PURGE
<br>http_access allow localhost PURGE
<br>http_access deny PURGE
<br>
<br># ACL lists
<br># ACL - Whitelist - User defined (whiteList)
<br>acl whiteList url_regex packages\.wazuh\.com
<br>acl whiteList url_regex archive\.ubuntu\.com
<br>acl whiteList url_regex security\.ubuntu\.com
<br>acl whiteList url_regex repos\.influxdata\.com
<br>
<br># ACL - Blacklist - User defined (blackList)
<br>acl blackList url_regex .*
<br>
<br># ACL - Remote fetched Blacklist (remoteblacklist)
<br>
<br># ACL - Block browser/user-agent - User defined (browser)
<br>
<br># ACL - SSL ports, default are configured in config.xml
<br># Configured SSL ports (if defaults are not listed, then they have been removed from the configuration!):
<br>acl SSL_ports port 443 # https
<br>
<br># Default Safe ports are now defined in config.xml
<br># Configured Safe ports (if defaults are not listed, then they have been removed from the configuration!):
<br># ACL - Safe_ports
<br>acl Safe_ports port 80 # http
<br>acl Safe_ports port 21 # ftp
<br>acl Safe_ports port 443 # https
<br>acl Safe_ports port 70 # gopher
<br>acl Safe_ports port 210 # wais
<br>acl Safe_ports port 1025-65535 # unregistered ports
<br>acl Safe_ports port 280 # http-mgmt
<br>acl Safe_ports port 488 # gss-http
<br>acl Safe_ports port 591 # filemaker
<br>acl Safe_ports port 777 # multiling http
<br>acl CONNECT method CONNECT
<br>
<br># ICAP SETTINGS
<br># disable icap
<br>icap_enable off
<br>
<br># Pre-auth plugins
<br>include /usr/local/etc/squid/pre-auth/*.conf
<br>
<br># Authentication Settings
<br>
<br># ACL list (Allow) whitelist
<br>http_access allow whiteList
<br>
<br></span></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace">
#
<br># ACL list (Deny) blacklist
<br>http_access deny blackList
<br>
<br># Google Suite Filter
<br>
<br># YouTube Filter
<br>
<br># Deny requests to certain unsafe ports
<br>
<br>http_access deny !Safe_ports <br># Deny CONNECT to other than secure SSL ports
<br>
<br>http_access deny CONNECT !SSL_ports <br>
<br># Only allow cachemgr access from localhost
<br>http_access allow localhost manager
<br>http_access deny manager
<br>
<br># We strongly recommend the following be uncommented to protect innocent
<br># web applications running on the proxy server who think the only
<br># one who can access services on "localhost" is a local user
<br>http_access deny to_localhost
<br>
<br># Auth plugins
<br>include /usr/local/etc/squid/auth/*.conf
<br>
<br>#
<br># Access Permission configuration:
<br>#
<br># Deny request from unauthorized clients
<br>
<br>#
<br># ACL - localnet - default these include ranges from selected interfaces (Allow local subnets)
<br>http_access allow localnet
<br>
<br># ACL - localhost
<br>http_access allow localhost
<br>
<br># Deny all other access to this proxy
<br>http_access deny all
<br># Post-auth plugins
<br>include /usr/local/etc/squid/post-auth/*.conf
<br>
<br># Caching settings
<br>cache_mem 1000 MB
<br>maximum_object_size 200 MB
<br>cache_replacement_policy heap LFUDA
<br>cache_dir ufs /var/squid/cache 100000 16 256
<br>
<br># Leave coredumps in the first cache dir
<br>coredump_dir /var/squid/cache
<br>
<br>#
<br># Add any of your own refresh_pattern entries above these.
<br>#
<br>
<br># Linux package cache:
<br>refresh_pattern pkg\.tar\.xz$ 0 20% 4320 refresh-ims
<br>refresh_pattern d?rpm$ 0 20% 4320 refresh-ims
<br>refresh_pattern deb$ 0 20% 4320 refresh-ims
<br>refresh_pattern udeb$ 0 20% 4320 refresh-ims
<br>refresh_pattern Packages\.bz2$ 0 20% 4320 refresh-ims
<br>refresh_pattern Sources\.bz2$ 0 20% 4320 refresh-ims
<br>refresh_pattern Release\.gpg$ 0 20% 4320 refresh-ims
<br>refresh_pattern Release$ 0 20% 4320 refresh-ims
<br># <a href="http://wiki.squid-cache.org/SquidFaq/WindowsUpdate">http://wiki.squid-cache.org/SquidFaq/WindowsUpdate</a>
<br>refresh_pattern -i <a href="http://microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip|esd)">microsoft.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip|esd)</a> 4320 80% 129600 reload-into-ims
<br>refresh_pattern -i <a href="http://windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip|esd)">windowsupdate.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip|esd)</a> 4320 80% 129600 reload-into-ims
<br>refresh_pattern -i <a href="http://windows.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip|esd)">windows.com/.*\.(cab|exe|ms[i|u|f]|[ap]sf|wm[v|a]|dat|zip|esd)</a> 4320 80% 129600 reload-into-ims
<br>
<br>refresh_pattern ^ftp: 1440 20% 10080
<br>refresh_pattern ^gopher: 1440 0% 1440
<br>refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
<br>refresh_pattern . 0 20% 4320
<br>
<br># Squid Options
<br># dns_v4_first reverses the order of preference to make Squid contact dual-stack websites over IPv4 first
<br>dns_v4_first on
<br>pinger_enable off
<br>access_log stdio:/var/log/squid/access.log squid
<br>cache_store_log stdio:/var/log/squid/store.log
<br># URI hanlding with Whitespaces (default=strip)
<br>uri_whitespace strip
<br># X-Forwarded header handling (default=on)
<br>forwarded_for on
<br># Disable squid logfile rotate to use system defaults
<br>logfile_rotate 0
<br># Define visible email
<br>cache_mgr admin@localhost.local
<br>error_directory /usr/local/etc/squid/errors/local<br></span></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><br></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace">Thanks<br></span></span></span></div><div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"><br></span></span></span></div><span style="font-family:monospace"><span style="font-family:monospace"><span style="font-family:monospace"></span></span></span></div></div>