[squid-users] Questions about Squid configuration

にば taku0919.taku at gmail.com
Thu Sep 5 09:01:20 UTC 2024 

I'm sorry. I forgot to attach the configuration file used in the
verification (partially masked).

2024年9月5日(木) 17:58 にば <taku0919.taku at gmail.com>:
>
> Hi Alex
>
> Thanks for the reply !
>
> I took the advice I received, reviewed the verification details, and
> verified again with the two recommended steps.
> The new verification includes the following four patterns:
> 1. successful communication of a valid request to an allowed site
> [command]
> curl https://pypi.org/ -v --cacert squid.crt -k
>
> 2. communication of invalid requests to allowed sites is denied
> [command] ※Invalid header host
> curl https://pypi.org/ -H "Host: www.yahoo.co.jp" -v --cacert squid.crt -k
>
> 3. communication of valid requests to prohibited sites is denied
> [command]
> curl https://www.yahoo.co.jp/ -v --cacert squid.crt -k
>
> 4. communication of invalid requests to prohibited sites is denied
> [command]
> curl https://www.yahoo.co.jp/ -H "Host:" -v --cacert squid.crt -k
>
>
> STEP１
> ＞1. Remove all of your current http_access rules. Keep ACLs. Perform
> ＞host_verify_strict and access tests to confirm that all valid requests
> ＞are denied and all invalid requests are rejected. If necessary, ask
> ＞questions, file bug reports, patch Squid, and/or adjust your
> ＞configuration to pass this test.
>
> ・Results for validation patterns
> １．403 Forbidden、X-Squid-Error: ERR_ACCESS_DENIED 0
> ２．403 Forbidden、X-Squid-Error: ERR_ACCESS_DENIED 0
> ３．403 Forbidden、X-Squid-Error: ERR_ACCESS_DENIED 0
> ４．403 Forbidden、X-Squid-Error: ERR_ACCESS_DENIED 0
>
> STEP２
> ＞2. Copy http_access rules, with comments, from generated
> ＞squid.conf.default. Insert your own access rules in the location marked
> ＞by "INSERT YOUR OWN RULE(S) HERE" comment. Perform host_verify_strict
> ＞and access tests to confirm that all valid requests to banned sites are
> ＞denied, all other valid requests are allowed, and all invalid requests
> ＞are rejected. If necessary, ask questions, file bug reports, patch
> ＞Squid, and/or adjust your configuration to pass this test.
>
> ・Results for validation patterns
> １．200 OK
> ２．409 Conflict、X-Squid-Error: ERR_CONFLICT_HOST 0
> ３．409 Conflict、X-Squid-Error: ERR_CONFLICT_HOST 0
> ４．200 OK
>
> So my question is, in STEP 1.
> ・I expected to get a different response for a valid request than for
> an invalid request, is this result correct?
> In STEP 2.
> ・4. still returns 200 OK. Is this due to an error in the existing
> configuration? Or do I need to add a new setting?
>
> I am not a good English, so I may not have understood Alex's advice exactly.
> If there are still mistakes, please point them out to me.
>
> Thank you in advance.
>
> 2024年8月30日(金) 22:27 Alex Rousskov <rousskov at measurement-factory.com>:
> >
> > On 2024-08-29 22:28, にば wrote:
> >
> > > With the newly reviewed configuration in the attachment
> >
> > OT: Please note that your configuration does not follow the recommended
> > http_access rules order template in squid.conf.default and might,
> > depending on your deployment environment, allow Squid to be used for
> > attacks on 3rd party resources (e.g., ssh services). This observation is
> > not related to your primary question and your "ban certain sites" goal.
> > Following suggestions at the end of this email should fix this problem.
> >
> >
> > > I found the following statement in the following official document
> > > https://www.squid-cache.org/Doc/config/host_verify_strict/
> > >
> > > ＞ * The host names (domain or IP) must be identical,
> > > ＞ but valueless or missing Host header disables all checks.
> > >
> > > So I ran an additional validation with an empty Host value, and the
> > > request succeeded for a domain that was not in the whitelist.
> > > The curl command for verification is below, and as before, only
> > > .pypi.org is allowed in the whitelist.
> > >
> > > date;curl https://www.yahoo.co.jp/ -H "Host:" -v --cacert squid.crt -k
> > >
> > > Is it possible for Squid to prevent such requests as well?
> >
> > Yes, a req_header ACL should be able to detect such requests (i.e.
> > requests without a Host header or with an empty Host header value).
> > However, I suspect that "missing Host" is _not_ the problem you should
> > be solving (as detailed below).
> >
> >
> > > I was able to confirm that any one of the SNI, IP, or Host in the
> > > request is incorrect (not whitelist allowed)
> > > and Squid will correctly check and return a 409 Conflict.
> >
> > IMHO, you should target/validate a different set of goals/conditions:
> >
> > * A valid request targeting a banned site should be denied (HTTP 403
> > response, %Ss=TCP_DENIED, %err_code=ERR_ACCESS_DENIED). This denial
> > should be triggered by an "http_access deny" rule, preferably an
> > explicit one. This denial will _not_ happen (and the request will
> > instead be forwarded to the banned site it targets) if you replace all
> > your http_access rules with a single "http_access allow all" line. This
> > denial does not depend on host_verify_strict and underlying code.
> >
> > * An invalid request should be rejected (HTTP 4xx response). This
> > includes, but is not limited to, host_verify_strict-driven rejections.
> > This rejection should happen even if you replace all your http_access
> > rules with a single "http_access allow all" line.
> >
> > AFAICT, your current configuration does not reach "deny valid requests
> > targeting banned sites" goal while your question implies that you are
> > incorrectly relying on host_verify_strict to perform that denial.
> >
> >
> > I recommend the following:
> >
> > 1. Remove all of your current http_access rules. Keep ACLs. Perform
> > host_verify_strict and access tests to confirm that all valid requests
> > are denied and all invalid requests are rejected. If necessary, ask
> > questions, file bug reports, patch Squid, and/or adjust your
> > configuration to pass this test.
> >
> > 2. Copy http_access rules, with comments, from generated
> > squid.conf.default. Insert your own access rules in the location marked
> > by "INSERT YOUR OWN RULE(S) HERE" comment. Perform host_verify_strict
> > and access tests to confirm that all valid requests to banned sites are
> > denied, all other valid requests are allowed, and all invalid requests
> > are rejected. If necessary, ask questions, file bug reports, patch
> > Squid, and/or adjust your configuration to pass this test.
> >
> >
> > HTH,
> >
> > Alex.
> >
> >
> > > 2024年8月8日(木) 21:33 Alex Rousskov <rousskov at measurement-factory.com>:
> > >>
> > >> On 2024-08-06 20:59, にば wrote:
> > >>
> > >>> When using Squid transparently, is it possible to control the
> > >>> whitelist of the domain to connect to and inspect the Host field in
> > >>> the request header together?
> > >>
> > >> Short answer: Yes.
> > >>
> > >>
> > >>> According to the verification results, the Host field can be inspected
> > >>> by "host_verify_strict on" in squid-transparent.conf, but it seems
> > >>> that the whitelist is not controlled.
> > >>
> > >> AFAICT, the configuration you have shared allows all banned[1] traffic
> > >> to/through https_port. For the problematic test case #5:
> > >>
> > >> All these http_access rules do _not_ match:
> > >>
> > >>> http_access allow localnet whitelist
> > >>> http_access deny localnet whitelist_https !https_port
> > >>> http_access deny localnet whitelist_transparent_https !https_port
> > >>
> > >>
> > >> And then this next rule matches and allows traffic through:
> > >>
> > >>> http_access allow https_port
> > >>
> > >>
> > >> This last http_access rule is not reached:
> > >>
> > >>> http_access deny all
> > >>
> > >>
> > >> N.B. The above analysis assumes that your https_port ACL is explicitly
> > >> defined in your squid.conf to match all traffic received at https_port.
> > >> If you do not have such an ACL defined, then you need to fix that
> > >> problem as well. I recommend naming ACLs differently from directive
> > >> names (e.g., "toHttpsPort" rather than "https_port").
> > >>
> > >>
> > >> Please note that Squid v4 is not supported by the Squid Project and is
> > >> very buggy. I recommend using Squid v6 or later.
> > >>
> > >>
> > >> HTH,
> > >>
> > >> Alex.
> > >> [1] Here, "banned" means "_not_ matching whitelist ACL".
> > >>
> > >>
> > >>> ■Configuration Details
> > >>> 〇squid-transparent.conf（Excerpts）
> > >>> #Whitelist
> > >>> acl whitelist dstdomain "/etc/squid/whitelist"
> > >>> acl whitelist dstdomain "/etc/squid/whitelist_transparent"
> > >>> acl whitelist_https dstdomain "/etc/squid/whitelist_https"
> > >>> acl whitelist_transparent_https dstdomain
> > >>> "/etc/squid/whitelist_transparent_https"
> > >>>
> > >>> proxy_protocol_access allow localnet
> > >>> proxy_protocol_access deny all
> > >>> http_access allow localnet whitelist
> > >>> http_access deny localnet whitelist_https !https_port
> > >>> http_access deny localnet whitelist_transparent_https !https_port
> > >>>
> > >>> # Handling HTTP requests
> > >>> http_port 3129 intercept
> > >>> # Handling HTTPS requests
> > >>> https_port 3130 intercept tcpkeepalive=60,30,3 ssl-bump
> > >>> generate-host-certificates=on dynamic_cert_mem_cache_size=20MB
> > >>> tls-cert=/etc/squid/ssl/squid.crt tls-key=/etc/squid/ssl/squid.key
> > >>> cipher=HIGH:MEDIUM:!LOW:!RC4:!SEED:!IDEA:!3DES:!MD5:!EXP:!PSK:!DSS
> > >>> options=NO_TLSv1,NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE
> > >>> tls-dh=prime256v1:/etc/squid/ssl/bump_dhparam.pem
> > >>> # Start up for squid process
> > >>> http_port 3131
> > >>> http_access allow https_port
> > >>> acl allowed_https_sites ssl::server_name "/etc/squid/whitelist"
> > >>> acl allowed_https_sites ssl::server_name "/etc/squid/whitelist_transparent"
> > >>> acl allowed_https_sites ssl::server_name "/etc/squid/whitelist_https"
> > >>> acl allowed_https_sites ssl::server_name
> > >>> "/etc/squid/whitelist_transparent_https"
> > >>>
> > >>> http_access deny all
> > >>>
> > >>> # strict setting
> > >>> host_verify_strict on
> > >>>
> > >>> # SSL_BUMP
> > >>> sslcrtd_program /usr/lib64/squid/security_file_certgen -s
> > >>> /var/lib/squid/ssl_db -M 20MB
> > >>> acl step1 at_step SslBump1
> > >>> acl step2 at_step SslBump2
> > >>> acl step3 at_step SslBump3
> > >>>
> > >>> ssl_bump bump all
> > >>
> > >>
> > >>> ■Verification of Settings
> > >>> I ran the curl command from each of the client environments that use Squid.
> > >>> 1. if SNI, Destination IP, and HeaderHost are correct, the user should
> > >>> be able to connect to pypi.org
> > >>> Command:
> > >>> date;curl https://pypi.org/ -v --cacert squid_2.crt -k
> > >>> Result: OK
> > >>>
> > >>> 2. rejection of communication to pypi.org if SNI is correct but
> > >>> destination IP and HeaderHost are incorrect
> > >>> Command:
> > >>> date;curl https://pypi.org/ --resolve pypi.org:443:182.22.24.252 -H
> > >>> "Host: www.yahoo.co.jp"  -v --cacert squid_2.crt -k
> > >>> Result: OK (409 Conflict is returned)
> > >>>
> > >>> 3. rejection of communication to pypi.org if SNI and destination IP
> > >>> are correct and HeaderHost is incorrect
> > >>> Command:
> > >>> date;curl https://pypi.org/ -H "Host: www.yahoo.co.jp" -v --cacert
> > >>> squid_2.crt -k
> > >>> Result: OK (409 Confilic returned)
> > >>>
> > >>> 4. rejection of communication to pypi.org if SNI and HeaderHost are
> > >>> correct but destination IP is incorrect
> > >>> Command:
> > >>> date;curl https://pypi.org/ --resolve pypi.org:443:182.22.24.252 -v
> > >>> --cacert squid_2.crt -k
> > >>> Result: OK (409 Confilic returned)
> > >>>
> > >>> 5. if SNI, destination IP, and HeaderHost are all invalid (yahoo.co.jp
> > >>> not registered in whitelist), communication will be rejected
> > >>> Command:
> > >>> date;curl https://yahoo.co.jp/ -v --cacert squid_2.crt -k
> > >>> Result: NG (301 Moved Permanently is returned, but it appears that the
> > >>> communication is reaching yahoo.co.jp)
> > >>
> > >>
> > >>
> > >>
> > >> _______________________________________________
> > >> squid-users mailing list
> > >> squid-users at lists.squid-cache.org
> > >> https://lists.squid-cache.org/listinfo/squid-users
> >
-------------- next part --------------
A non-text attachment was scrubbed...
Name: step2.squid-transparent.conf
Type: application/octet-stream
Size: 4044 bytes
Desc: not available
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20240905/22c0ff38/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: step1.squid-transparent.conf
Type: application/octet-stream
Size: 3430 bytes
Desc: not available
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20240905/22c0ff38/attachment-0003.obj>

More information about the squid-users mailing list