[squid-users] Unable to access internal resources via hostname

Mon Sep 30 17:22:42 UTC 2024

Hey Guys, 

I wanted to follow-up on this. 

The answer was simply adding the "append_domain" directive to my config. 

Everything is good now, thank you for all the help. 

- Josh 

-----Original Message-----
From: squid-users <squid-users-bounces at lists.squid-cache.org> On Behalf Of Amos Jeffries
Sent: Saturday, September 7, 2024 10:51 AM
To: squid-users at lists.squid-cache.org
Subject: Re: [squid-users] Unable to access internal resources via hostname

Caution: This email originated from outside of Hexcel. Do not click links or open attachments unless you recognize the sender and know the content is safe.

On 6/09/24 03:56, Piana, Josh wrote:
> Hello Amos,
>
> While the comments did say that it was just the 10.46.11.0 range, I don't think there's any other ACL forcing that. I tried adding the the two internal sites that are being blocked by their IP, restarted Squid, and tested. Still being blocked. You are right though, both of those web addresses are on a different IP scheme. Ideally we want anything on 172.0.0.0 to be allowed, and 10.96.0.0. The other question I have is, even if we specify those sites IP as "allowed", shouldn't we be able to brwose to them by their hostname as well?

That depends no your configuration policy. The default squid.conf only checks that clients are from the LAN and not doing any nasty protocol trick attacks.

>
> Currently, those internal sites ARE reachable. But only if we use IP. While this doesn't bother me, personally, the rest of our users would like to keep browsing via hostname as that's what they're used to and what many have shortcuts for.
>

Currently I see the ACL "local_dst_dom" is commented out (disabled).

I guess you also have not listed the hostname or domain in the file loaded by "authless_dst" ACL.

> In regards to the results of /etc/resolv.conf, see below:
> search ad.arc-tech.com
> nameserver 10.46.11.67
>

Okay. Then the "NDOT" default of 1 will be applied. So for these HTTP
messages:

   CONNECT hexp:443 HTTP/1.1
   Host: hexp
   ...

Will be interpreted by Squid as URL:

  https://hexp/

The "dstdomain" ACL will try to match "hexp" exactly.

The "dst" ACL will try to match IPs of "hexp.ad.arc-tech.com"

> There must be a better way to just allow internal to internal traffic without needing to authenticate through the web proxy. The old config had it, but that was part of the issue. We have no idea how that was working, it didn't make sense at all and it was a bit outdated, Version 2.5 as opposed to our current 5.5.
>
> I'm happy to post out config again here, as it's changed a bit and I have cleaned it up.
>
> # squid.conf - Squid web cache configuration
>
> ######################################################################
> ########
> # General
> ######################################################################
> ########
>
> # 2020MAR23 running out with just 1024 as we switch to Hexcel.com OMA 
> max_filedesc 4096
>

Unrelated to your problem, but FYI this should probably be much larger (ie 64K minimum, up to 100x expected user count) for a production proxy.

>
> ######################################################################
> ########
> # Logging
> ######################################################################
> ########
>
> # this makes the logs readable to humans logformat custom %tl.%03tu 
> %>a %Ss/%03>Hs %<st %rm %ru %[un %Sh/%<a %mt access_log 
> daemon:/var/log/squid/access.log custom
>
> # this gives better error reporting
> logformat custom %err_code/%err_detail access_log 
> daemon:/var/log/squid/access.log custom
>

This repeat of "custom" will cause issues.

IIRC this was added misunderstanding of Alex instructions.
What he meant was to **add** "%err_code/%err_detail" to your existing "custom" format.

Like this:

logformat custom %tl.%03tu %>a %Ss/%03>Hs %<st %rm %ru %[un \
     %Sh/%<a %mt %err_code/%err_detail

> # Red Hat-ish log names
> cache_log /var/log/squid/cache.log
> cache_access_log /var/log/squid/access.log

This setting opens a third logger writing to access.log, causing more issues.

Remove this "cache_access_log" line.

>
> # store_log is only useful for debugging cache_store_log none

FYI, off by default on current Squid. You can probably erase this setting entirely now.

> ######################################################################
> ########
> # Network - General/misc
> ######################################################################
> ########
>
> # our HTTP proxy port
> http_port 10.46.11.69:8080
> # loopback management
> http_port 127.0.0.1:3128

FWIW, you have denied access to "dst 127.0.0.0/8". So traffic to this port will be rejected.

>
> # disable ICP, port is typically 3130
> icp_port 0

FYI; disabled by default in modern Squid. You can remove "icp_port".

>
> # if set to "on", Squid will append your client's IP address in the 
> HTTP requests it forwards forwarded_for off

"off" will send the text "unknown".

It is better to use "transparent" (pass-thru unchanged) or "delete"
(erase if existing).

> ######################################################################
> ########
> # Authentication
> ######################################################################
> ########
>
> auth_param negotiate program /usr/lib64/squid/negotiate_kerberos_auth 
> -k /etc/squid/HTTP.keytab -s 
> HTTP/ARCGATE2.ad.arc-tech.com at AD.ARC-TECH.COM
> auth_param negotiate children 10
> auth_param negotiate keep_alive on
> acl kerb-auth proxy_auth REQUIRED
>
> ######################################################################
> ######## # Access control - shared/common ACL definitions 
> ######################################################################
> ########
>
> # 
> ----------------------------------------------------------------------
> ------ # networks and hosts (by name or IP address)
>
> # acl all src all
>
> acl src_self src 127.0.0.0/8
> acl src_self src 10.46.11.69
>
> acl dst_self dst 127.0.0.0/8
> acl dst_self dst 10.46.11.69

FYI, there are more IPs than just 127.0.0.0/8 which can be problematic as loopback sources.

Replace "dst_self" with:

  acl to_localhost dst 10.46.11.69

>
> acl from_arc src 10.46.0.0/15
>
> acl local_dst_addr dst 10.0.0.0/8
> acl local_dst_addr dst 172.0.0.0/8
>
> # 
> ----------------------------------------------------------------------
> ------
> # protocols (URL schemes)
>
> acl proto_FTP proto FTP
> acl proto_HTTP proto HTTP
>
> # 
> ----------------------------------------------------------------------
> ------
> # TCP port numbers
>
> # TCP ports for ordinary HTTP
> acl http_ports port 80                # standard HTTP
> acl http_ports port 81                # common alternative
> acl http_ports port 8001      # epson.com support sub-site
> acl http_ports port 8080      # common alternative

Other ports that are common:

  acl http_ports port 88 8000 8888 # ad-hoc services
  acl http_ports port 1080  # SOCK frontend to HTTP service
  acl http_ports port 21-22 # http:// frontend to FTP service
  acl http_ports port 443   # https:// URLs

Or really, **any** port can be valid now that Alt-Svc is widely used.

>
> # TCP ports for HTTP-over-SSL
> acl Ssl_ports port 443
> acl Ssl_ports port 9571               # lexmark.com
>
> # TCP ports for plain FTP command channel acl ftp_ports port 21
>
> # TCP ports for SSH/SFTP (secure shell) acl ssh_ports port 22

FTR, "ssh_ports" is unused. SSH and SFTP can only use Squid via CONNECT tunnel anyway. So you should list this port as part of "Ssl_ports".

>
> # 
> ----------------------------------------------------------------------
> ------
> # HTTP methods (and pseudo-methods)
>
> acl method_CONNECT method CONNECT

FYI: we have a built-in for modern Squid:

   acl CONNECT method CONNECT

You can replace all "method_CONNECT" with just "CONNECT".

>
> # list of standard HTTP methods
> acl methods_std method GET HEAD POST PUT DELETE acl methods_std method 
> TRACE OPTIONS
>

Consider removing these ACLs and the extra complexity they create.
Modern HTTP considers many other method names "standard" and current Squid follows the standard-required handling for all of them, and special ones.

> ######################################################################
> #######
> # Access control - general proxy
> ######################################################################
> ########
>
> # This major section is about which HTTP proxy clients can use Squid # 
> as an HTTP proxy server.  As opposed to the maintenance/admin-type # 
> stuff in the previous section.
>
> # 
> ----------------------------------------------------------------------
> ------
> # basic deny rules
> # these block stuff that's never good
> # we put most denies after user-auth so we know *who* is trying
>

Missing basic security to prevent DoS and protocol hijacking attacks:

  http_access deny !Safe_ports
  http_access deny CONNECT !Ssl_ports

You can define Safe_ports like this:

   acl Safe_Ports any-of http_ports Ssl_ports ftp_ports

> # block attempts to connect to proxy server via proxy http_access deny 
> dst_self
>

Use:
   http_access deny to_localhost

> # block clients which are the proxy server machine itself http_access 
> deny src_self
>

FYI, there should be nothing wrong with other software running on the proxy machine (eg software updaters) using Squid. They have to obey the same rules an any other client.

> # deny anything not from the LAN
> http_access deny !from_arc

FYI, we use "localnet" ACL to define the LAN. Makes it a bit easier for others to assist if you use the same. Up to you though.

>
> # 
> ----------------------------------------------------------------------
> ------
> # allow without authentication
> # these rules allow certain connects without user authentication # 
> these allow any protocol/method/etc
>
> #                 ***** IMPORTANT *****
> # Adding to these lists also exempts from all content filtering.
> # In particular, executables will be allowed to download!
> #                 ***** IMPORTANT *****
>
> # allow connects to local destinations without authentication # by 
> domain name from URL # acl local_dst_dom dstdomain ad.arc-tech.com
> # http_access       allow local_dst_dom
> # http_reply_access allow local_dst_dom

You have a domain name whitelist ACL two sections below.
You can remove the above, and list and use the later one for both
"local" and other domains.

>
> # by IP address name resolves to
> http_access       allow local_dst_addr
> # http_reply_access allow local_dst_addr
>

Maybe rename "local_dst_addr" to "whitelist_dst_ips" for clarity on what
it is doing.

Which brings up a security issue: When you list the entire LAN range in
this ACL, anyone accessing the proxy is allowed to do whatever they want
to your LAN machines. That includes all LAN **and** WAN connections.

Additionally, the bypass of authentication leaves you no trace that this
was actually a user, and not some infected LAN machine spreading malware
across your LAN. Which is rather risky.

Prefer to keep by-IP whitelist's empty or minimal content.

> # allow trusted hosts without authentication
> # these are just ip's on the 10.46.11.x network
> acl authless_src src "/etc/squid/authless_src"
> http_access       allow authless_src
> # http_reply_access allow authless_src
>
> # allow the following destinations without authentication
> # list of random approved websites
> # whats the advantage of not authenticating?
> acl authless_dst dstdomain "/etc/squid/authless_dst"
> http_access       allow authless_dst
> # http_reply_access allow authless_dst
>

This should be combined with "local_dst_dom". Just use different lines
like so:

  acl authless_dst dstdomain "/etc/squid/authless_dst"
  acl authless_dst dstdomain .ad.arc-tech.com
  acl authless_dst dstdomain .hexcelssp.com hexcelssp

  http_access allow whitelist

> # ----------------------------------------------------------------------------
> # block before authentication
> # these rules block certain connects without user authentication
> # done for software which handles proxy auth requests badly
> # for example, popping up many auth prompts
> # this does mean we cannot whitelist for users
>
> # blocked destinations, by host or domain, before authentication
> # websites that are auto-deny
> acl bad_domains_preauth dstdomain "/etc/squid/bad_domains_preauth"
> http_access deny bad_domains_preauth
>
> # ----------------------------------------------------------------------------
> # require proxy authentication
>
> # ********************************************************************
> # * anything past this point requires users to authenticate to proxy *
> # ********************************************************************
>
> # Uncomment these lines to disable authentication requirement for all
> # but a few test boxes.  Useful if NTLM gets broken.
> # acl from_test_boxes src 10.2.1.5
> # http_access allow !from_test_boxes from_arc
>

FYI, you have "deny !from_arc" earlier above.

This is simpler and does the same thing as the above line:

   # http_access allow !from_test_boxes

> # block clients which are not authenticated
> # http_access deny !authenticated
>

You named the ACL "kerb-auth", not "authenticated".

> # block certain user IDs from using proxy server
> # list of ad users and service accounts to automatically deny proxy
> acl block_user proxy_auth_regex -i "/etc/squid/block_user"
> http_access deny block_user

I suggest this to prevent re-login loop for these forbidden services:

  http_access deny !kerb-auth
  http_access deny block_user all

>
> # ----------------------------------------------------------------------------
> # general whitelist
>
> # whitelist for locations/sites (override blocking)
> # allows complex URLs
> # some bad_urls patterns are generic
> # they block suspicous URLs or generally unwanted sites
> # this whitelist can make specific exceptions within those
> acl bad_exception_urls url_regex -i "/etc/squid/bad_exception_urls"
>
> # ----------------------------------------------------------------------------
> # executable blocking
>
> # we do this separately from other content blocking
> # in order to allow ITLIB to still download
> # doing it with other blocking causes a flood of browser auth req's
> # for users visting sites with video
>
> # file extensions to block
> # don't try to block .COM files here
> # it will get confused with .com domain and block too many things
> acl exec_files url_regex -i "/etc/squid/exec_files"

Consider using the "urlpath_regex" ACL type instead. It does not match
against the scheme://domain:port part of URLs.

>
> # usernames not to block
> acl exec_users proxy_auth_regex -i "/etc/squid/exec_users"
>
> # activate
> http_access deny !bad_exception_urls !exec_users exec_files
> deny_info ERR_BLOCK_TYPE exec_files
>
> # ----------------------------------------------------------------------------
> # selective whitelists
>
> # these override the general blacklists by explictly allowing things
> # some multimedia services (I.E. YouTube) are permitted for certain users
> # exempts them from content blocking in this section
>
> # list of users for this
> acl mmedia_users proxy_auth_regex -i "/etc/squid/mmedia_users"
>
> # list of sites for this
> acl mmedia_sites dstdomain "/etc/squid/mmedia_sites"
>
> # if a mmedia user attempts to access a mmedia site via appropriate protocols, allow it
> # done for both HTTP/TCP/80 and HTTP/SSL/443
> # done for both HTTP request as well as HTTP reply/response
> # otherwise the later bad_types Content-Type multimedia blocking rule kicks in
>
> http_access       allow methods_std    proto_HTTP http_ports mmedia_sites mmedia_users
> # tp_reply_access allow methods_std    proto_HTTP http_ports mmedia_sites mmedia_users
>
> http_access       allow method_CONNECT            ssl_ports  mmedia_sites mmedia_users
> # http_reply_access allow method_CONNECT            ssl_ports  mmedia_sites mmedia_users

FYI, Websites like YouTube can dynamically negotiate the media source
ports to be non-standard values using Alt-Svc headers.

If you had the default security settings at the top of your http_access
rules:

  http_access deny !Safe_ports
  http_access deny CONNECT !SSL_Ports

... you would not need to make (bad) assumptions about the protocols and
ports used by media sites here.

Also, "mmedia_users" being last on the line risks a 407 re-auth
challenge occuring. Best place it earlier.

Leaving these rules as:

    http_access allow mmedia_users mmedia_sites

>
> # ----------------------------------------------------------------------------
> # general blacklists
> # porn, wastes of bandwidth, etc.
>
>
> # blocked destinations, by host or domain
> # sites we don't want people using
> acl bad_domains dstdomain "/etc/squid/bad_domains"
> http_access deny !bad_exception_urls bad_domains
> deny_info ERR_BLOCK_DST       bad_domains
>
> # blocked destinations, by host or domain, regex pattern
> acl bad_domains_regex dstdom_regex -i "/etc/squid/bad_domains_regex"
> http_access deny !bad_exception_urls bad_domains_regex
> deny_info ERR_BLOCK_DST       bad_domains_regex
>
> # blocked destinations, by complex URL
> # typical use: block just part of a site, by URL path
> # example use: block just the advertsing section of a site
> acl bad_urls url_regex -i "/etc/squid/bad_urls"
> http_access deny !bad_exception_urls bad_urls
> deny_info ERR_BLOCK_DST       bad_urls
>
> # blocked content types, by apparent file name
> acl bad_files urlpath_regex -i "/etc/squid/bad_files"
> http_access deny !bad_exception_urls bad_files
> deny_info ERR_BLOCK_TYPE bad_files
>
> # blocked content types, by MIME content type, in response
> acl bad_types rep_mime_type -i "/etc/squid/bad_types"
> # http_reply_access deny bad_types !bad_exception_urls
> deny_info ERR_BLOCK_TYPE bad_types

FYI, in order for the deny_info action to be taken the ACL it is
attached to must be the final one on a line.

Previous config was fine, but this "bad_types" use will not work. You
need to re-order the access checks like this:

  # http_reply_access deny !bad_exception_urls bad_types

>

This part from here ...

> # ----------------------------------------------------------------------------
> # standard web (HTTP PUT/GET/etc) access
>
> # We basically allow any auth'ed user to connect via HTTP to anywhere,
> # so long as it uses a standard port, and is not a direct CONNECT
> # attempt, or blocked, or any of the other stuff above.
>
> http_access allow http_ports proto_HTTP methods_std
>
> # ----------------------------------------------------------------------------
> # direct CONNECT for HTTP-over-SSL (HTTPS)
>
> http_access allow method_CONNECT ssl_ports
>
> # ----------------------------------------------------------------------------
> # deny any other CONNECT attempts
>
> http_access deny method_CONNECT
>
> # ----------------------------------------------------------------------------
> # FTP via HTTP proxy
>
> http_access allow ftp_ports proto_FTP
>

... to here.

Is better handled by the default squid.conf:

  http_access deny !Safe_ports
  http_access deny CONNECT !Ssl_ports

Placed at the top of your http_access lines.

> # ----------------------------------------------------------------------------
> # catch-all defaults
> http_access allow kerb-auth
>
> # deny any request we missed in the above
> http_access deny all
>
> # If we allowed the request, allow the reply (HTTP response) as well.
> # Rules above many allow or deny specific reply before now.
> # If nothing more specific matched, we allow.
> # This should be OK since we filter mainly on requests.
> # http_reply_access allow all
>

FWIW, it is hard to tell what the exact last reply action could have
been. So I would un-comment this allow line to be certain it is what
happens.

> ##############################################################################
> # END OF FILE
> ##############################################################################
>

HTH
Amos

_______________________________________________
squid-users mailing list
squid-users at lists.squid-cache.org
https://lists.squid-cache.org/listinfo/squid-users