[squid-users] No failover when default parent proxy fails (Squid 3.5.12)

Jens Offenbach wolle5050 at gmx.de
Thu Mar 16 09:39:55 UTC 2017


This is the sceanrio;

Squid 3.5.12 is installed on "squid-proxy.mycompany.com". The two parent proxies are:
- Primary: proxy.mycompany.de:8080 (139.2.1.3)
- Fallback: roxy.mycompany.de:8080 (139.2.1.4)

I have misunderstood the "default" option in "cache_peer". When I got it right, it has the meaning of a fallback, so I switched it to "roxy.mycompany.de". "proxy.mycompany.de" should always be used and "roxy.mycompany.de" only when "proxy.mycompany.de" fails.

squid.conf:

# ACCESS CONTROLS
# -----------------------------------------------------------------------------
  # Local Networks
  acl localnet src 139.2.0.0/16
  acl localnet src 193.96.112.0/21
  acl localnet src 192.109.216.0/24
  acl localnet src 100.1.4.0/22
  acl localnet src 10.0.0.0/8
  acl localnet src 172.16.0.0/12
  acl localnet src 192.168.0.0/16

  # Materna Networks
  acl to_matnet dst 139.2.0.0/16
  acl to_matnet dst 193.96.112.0/21
  acl to_matnet dst 192.109.216.0/24
  acl to_matnet dst 100.1.4.0/22
  acl to_matnet dst 10.0.0.0/8
  acl to_matnet dst 172.16.0.0/12
  acl to_matnet dst 192.168.0.0/16

  # SSL-Ports
  acl SSL_ports port 443 # https
  acl SSL_ports port 563 # snews
  acl SSL_ports port 873 # rsync

  # Safe-Ports
  acl Safe_ports port 80  # http
  acl Safe_ports port 21  # ftp
  acl Safe_ports port 443 # https
  acl Safe_ports port 70  # gopher
  acl Safe_ports port 210 # wais
  acl Safe_ports port 1025-65535 # unregistered ports
  acl Safe_ports port 280 # http-mgmt
  acl Safe_ports port 488 # gss-http
  acl Safe_ports port 591 # filemaker
  acl Safe_ports port 777 # multiling http
  acl Safe_ports port 631 # cups
  acl Safe_ports port 873 # rsync
  acl Safe_ports port 901 # SWAT

  # HTTPS
  acl CONNECT method CONNECT

  http_access deny !Safe_ports
  http_access deny CONNECT !SSL_ports
  http_access allow manager localhost
  http_access deny  manager
  http_access allow localnet
  http_access allow localhost
  http_access deny all

# NETWORK OPTIONS
# -----------------------------------------------------------------------------
  http_port 3128
  http_port 3129 intercept

# OPTIONS WHICH AFFECT THE NEIGHBOR SELECTION ALGORITHM
# -----------------------------------------------------------------------------
  cache_peer proxy.materna.de parent 8080 0 no-digest no-query connect-timeout=5 connect-fail-limit=2
  cache_peer  roxy.materna.de parent 8080 0 no-digest no-query connect-timeout=5 connect-fail-limit=2 default

# MEMORY CACHE OPTIONS
# -----------------------------------------------------------------------------
  maximum_object_size_in_memory 8 MB
  memory_replacement_policy heap LFUDA
  cache_mem 256 MB

# DISK CACHE OPTIONS
# -----------------------------------------------------------------------------
  maximum_object_size 10 GB
  cache_replacement_policy heap GDSF
  cache_dir ufs /var/cache/squid 88894 16 256 max-size=10737418240

# LOGFILE OPTIONS
# -----------------------------------------------------------------------------
  access_log daemon:/var/log/squid/access.log squid

# OPTIONS FOR TROUBLESHOOTING
# -----------------------------------------------------------------------------
  cache_log /var/log/squid/cache.log
  coredump_dir /var/log/squid
  debug_options 44,2

# OPTIONS FOR TUNING THE CACHE
# -----------------------------------------------------------------------------
  max_stale 6 days
  shutdown_lifetime 5 seconds

# ADMINISTRATIVE PARAMETERS
# -----------------------------------------------------------------------------
  visible_hostname proxy.materna.com

# OPTIONS INFLUENCING REQUEST FORWARDING 
# -----------------------------------------------------------------------------
  always_direct allow to_matnet
  never_direct  allow all

# DNS OPTIONS
# -----------------------------------------------------------------------------
  dns_nameservers 139.2.34.171
  dns_nameservers 139.2.34.37

# MISCELLANEOUS
# -----------------------------------------------------------------------------
  memory_pools off

Now, I block traffic on "squid-proxy.mycompany.com" to the primary proxy "proxy.mycompany.de" (139.2.1.3) using IPTables:
$ iptables -A OUTPUT -p icmp -d 139.2.1.3 -j DROP
$ iptables -A OUTPUT -p tcp -d 139.2.1.3 -j DROP
$ iptables -A OUTPUT -p udp -d 139.2.1.3 -j DROP

On the test machine, I use:
$ export http_proxy=http://squid-proxy.mycompany.com:3128/
$ export https_proxy=http://squid-proxy.mycompany.com:3128/
$ export HTTP_PROXY=http://squid-proxy.mycompany.com:3128/
$ export HTTPS_PROXY=http://squid-proxy.mycompany.com:3128/

Trying to download a resource:
$ wget https://repository.apache.org/content/groups/snapshots/org/apache/karaf/apache-karaf/4.1.1-SNAPSHOT/apache-karaf-4.1.1-20170315.084054-35.tar.gz

The download hangs for 2 minutes until it gets started. A retry shows the same results, the download starts after 2 minutes showing:
--2017-03-16 09:31:26--  https://repository.apache.org/content/groups/snapshots/org/apache/karaf/apache-karaf/4.1.1-SNAPSHOT/apache-karaf-4.1.1-20170314.154157-34.tar.gz
Resolving squid-proxy.mycompany.com (squid-proxy.mycompany.com)... 10.152.132.41
Connecting to squid-proxy.mycompany.com (squid-proxy.mycompany.com)|10.152.132.41|:3128... connected.

cache.log:

2017/03/16 10:17:47 kid1| Shutdown: NTLM authentication.
2017/03/16 10:17:47 kid1| Shutdown: Negotiate authentication.
2017/03/16 10:17:47 kid1| Shutdown: Digest authentication.
2017/03/16 10:17:47 kid1| Shutdown: Basic authentication.
CPU Usage: 0.084 seconds = 0.052 user + 0.032 sys
Maximum Resident Size: 113840 KB
Page faults with physical i/o: 0
2017/03/16 10:17:48 kid1| Starting Squid Cache version 3.5.12 for x86_64-pc-linux-gnu...
2017/03/16 10:17:48 kid1| Service Name: squid
2017/03/16 10:17:48| pinger: Initialising ICMP pinger ...
2017/03/16 10:18:09.579 kid1| 44,2| peer_select.cc(258) peerSelectDnsPaths: Find IP destination for: http://proxy.materna.de:8080/squid-internal-dynamic/netdb' via proxy.materna.de
2017/03/16 10:18:09.579 kid1| 44,2| peer_select.cc(280) peerSelectDnsPaths: Found sources for 'http://proxy.materna.de:8080/squid-internal-dynamic/netdb'
2017/03/16 10:18:09.579 kid1| 44,2| peer_select.cc(281) peerSelectDnsPaths:   always_direct = ALLOWED
2017/03/16 10:18:09.579 kid1| 44,2| peer_select.cc(282) peerSelectDnsPaths:    never_direct = DUNNO
2017/03/16 10:18:09.579 kid1| 44,2| peer_select.cc(286) peerSelectDnsPaths:          DIRECT = local=0.0.0.0 remote=139.2.1.3:8080 flags=1
2017/03/16 10:18:09.579 kid1| 44,2| peer_select.cc(295) peerSelectDnsPaths:        timedout = 0
2017/03/16 10:18:12.279 kid1| 44,2| peer_select.cc(258) peerSelectDnsPaths: Find IP destination for: http://roxy.materna.de:8080/squid-internal-dynamic/netdb' via roxy.materna.de
2017/03/16 10:18:12.279 kid1| 44,2| peer_select.cc(280) peerSelectDnsPaths: Found sources for 'http://roxy.materna.de:8080/squid-internal-dynamic/netdb'
2017/03/16 10:18:12.279 kid1| 44,2| peer_select.cc(281) peerSelectDnsPaths:   always_direct = ALLOWED
2017/03/16 10:18:12.279 kid1| 44,2| peer_select.cc(282) peerSelectDnsPaths:    never_direct = DUNNO
2017/03/16 10:18:12.279 kid1| 44,2| peer_select.cc(286) peerSelectDnsPaths:          DIRECT = local=0.0.0.0 remote=139.2.1.4:8080 flags=1
2017/03/16 10:18:12.279 kid1| 44,2| peer_select.cc(295) peerSelectDnsPaths:        timedout = 0
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(258) peerSelectDnsPaths: Find IP destination for: repository.apache.org:443' via proxy.materna.de
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(258) peerSelectDnsPaths: Find IP destination for: repository.apache.org:443' via proxy.materna.de
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(258) peerSelectDnsPaths: Find IP destination for: repository.apache.org:443' via roxy.materna.de
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(258) peerSelectDnsPaths: Find IP destination for: repository.apache.org:443' via roxy.materna.de
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(280) peerSelectDnsPaths: Found sources for 'repository.apache.org:443'
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(281) peerSelectDnsPaths:   always_direct = DENIED
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(282) peerSelectDnsPaths:    never_direct = ALLOWED
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(292) peerSelectDnsPaths:      cache_peer = local=0.0.0.0 remote=139.2.1.3:8080 flags=1
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(292) peerSelectDnsPaths:      cache_peer = local=0.0.0.0 remote=139.2.1.3:8080 flags=1
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(292) peerSelectDnsPaths:      cache_peer = local=0.0.0.0 remote=139.2.1.4:8080 flags=1
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(292) peerSelectDnsPaths:      cache_peer = local=0.0.0.0 remote=139.2.1.4:8080 flags=1
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(295) peerSelectDnsPaths:        timedout = 0

access.log

1489656077.628 159679 10.30.216.160 TCP_TUNNEL/200 26328966 CONNECT repository.apache.org:443 - ANY_OLD_PARENT/139.2.1.4 -


Any hints?

Jens
 

Gesendet: Donnerstag, 16. März 2017 um 09:27 Uhr
Von: "Amos Jeffries" <squid3 at treenet.co.nz>
An: squid-users at lists.squid-cache.org
Betreff: Re: [squid-users] No failover when default parent proxy fails (Squid 3.5.12)
On 16/03/2017 7:05 p.m., Jens Offenbach wrote:
> Thanks for your quick response...
>
> I have also configured, but the value seems not to be honored:
> connect_timeout 30 seconds
>
> The primary peer is down, but Squid does not print any "Dead parent"
> in the logs. Every HTTPS request is forwarded to the primary peer and
> it takes 1 minute until the secondary peer gets used, even with
> "connect_timeout 30 seconds". I think, I am facing the first issue
> that has been fixed by your patch.
>

The global config options being ignored completely is correct because
your peer have individual connect-timeout=5 settings.

So, those 5sec timeouts should be used instead now as before.

Though note that they apply only to how long a TCP connection (SYN,
SYN-ACK) is waited for. There is also a dns_timeout and peer selection
timeout that apply separately to the act of connecting. And a
forward_timeout global limit that all those operations have to fit
within, including retries.


Did you have a chance to try the debug setting I suggested at the
beginning? That will give you an immediate view about what Squid is
detecting as usable paths for each and every request and at what times
relative to the DEAD/LIVE notice.



> Are there any plans to backport this fix to Xenial APT repositories
> or to create a new Debian package for Squid4/5?

That is up to the Ubuntu server team, but I think it Unlikely. Zesty is
the current stable and things like this generally dont have enough
widespread impact to qualify for LTS backports.

Debian is now frozen to stabilize for the "Buster" release, that will
contain Squid-3.5.23 plus some few critical patches which are already
set. A Squid-4 package is ready and waiting for the release freeze to
end before it goes public in the Debian Unstable/Testing repos.

Amos

_______________________________________________
squid-users mailing list
squid-users at lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users


More information about the squid-users mailing list