[squid-users] No failover when default parent proxy fails (Squid 3.5.12)
Jens Offenbach
wolle5050 at gmx.de
Thu Mar 16 09:39:55 UTC 2017
This is the sceanrio;
Squid 3.5.12 is installed on "squid-proxy.mycompany.com". The two parent proxies are:
- Primary: proxy.mycompany.de:8080 (139.2.1.3)
- Fallback: roxy.mycompany.de:8080 (139.2.1.4)
I have misunderstood the "default" option in "cache_peer". When I got it right, it has the meaning of a fallback, so I switched it to "roxy.mycompany.de". "proxy.mycompany.de" should always be used and "roxy.mycompany.de" only when "proxy.mycompany.de" fails.
squid.conf:
# ACCESS CONTROLS
# -----------------------------------------------------------------------------
# Local Networks
acl localnet src 139.2.0.0/16
acl localnet src 193.96.112.0/21
acl localnet src 192.109.216.0/24
acl localnet src 100.1.4.0/22
acl localnet src 10.0.0.0/8
acl localnet src 172.16.0.0/12
acl localnet src 192.168.0.0/16
# Materna Networks
acl to_matnet dst 139.2.0.0/16
acl to_matnet dst 193.96.112.0/21
acl to_matnet dst 192.109.216.0/24
acl to_matnet dst 100.1.4.0/22
acl to_matnet dst 10.0.0.0/8
acl to_matnet dst 172.16.0.0/12
acl to_matnet dst 192.168.0.0/16
# SSL-Ports
acl SSL_ports port 443 # https
acl SSL_ports port 563 # snews
acl SSL_ports port 873 # rsync
# Safe-Ports
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 631 # cups
acl Safe_ports port 873 # rsync
acl Safe_ports port 901 # SWAT
# HTTPS
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow manager localhost
http_access deny manager
http_access allow localnet
http_access allow localhost
http_access deny all
# NETWORK OPTIONS
# -----------------------------------------------------------------------------
http_port 3128
http_port 3129 intercept
# OPTIONS WHICH AFFECT THE NEIGHBOR SELECTION ALGORITHM
# -----------------------------------------------------------------------------
cache_peer proxy.materna.de parent 8080 0 no-digest no-query connect-timeout=5 connect-fail-limit=2
cache_peer roxy.materna.de parent 8080 0 no-digest no-query connect-timeout=5 connect-fail-limit=2 default
# MEMORY CACHE OPTIONS
# -----------------------------------------------------------------------------
maximum_object_size_in_memory 8 MB
memory_replacement_policy heap LFUDA
cache_mem 256 MB
# DISK CACHE OPTIONS
# -----------------------------------------------------------------------------
maximum_object_size 10 GB
cache_replacement_policy heap GDSF
cache_dir ufs /var/cache/squid 88894 16 256 max-size=10737418240
# LOGFILE OPTIONS
# -----------------------------------------------------------------------------
access_log daemon:/var/log/squid/access.log squid
# OPTIONS FOR TROUBLESHOOTING
# -----------------------------------------------------------------------------
cache_log /var/log/squid/cache.log
coredump_dir /var/log/squid
debug_options 44,2
# OPTIONS FOR TUNING THE CACHE
# -----------------------------------------------------------------------------
max_stale 6 days
shutdown_lifetime 5 seconds
# ADMINISTRATIVE PARAMETERS
# -----------------------------------------------------------------------------
visible_hostname proxy.materna.com
# OPTIONS INFLUENCING REQUEST FORWARDING
# -----------------------------------------------------------------------------
always_direct allow to_matnet
never_direct allow all
# DNS OPTIONS
# -----------------------------------------------------------------------------
dns_nameservers 139.2.34.171
dns_nameservers 139.2.34.37
# MISCELLANEOUS
# -----------------------------------------------------------------------------
memory_pools off
Now, I block traffic on "squid-proxy.mycompany.com" to the primary proxy "proxy.mycompany.de" (139.2.1.3) using IPTables:
$ iptables -A OUTPUT -p icmp -d 139.2.1.3 -j DROP
$ iptables -A OUTPUT -p tcp -d 139.2.1.3 -j DROP
$ iptables -A OUTPUT -p udp -d 139.2.1.3 -j DROP
On the test machine, I use:
$ export http_proxy=http://squid-proxy.mycompany.com:3128/
$ export https_proxy=http://squid-proxy.mycompany.com:3128/
$ export HTTP_PROXY=http://squid-proxy.mycompany.com:3128/
$ export HTTPS_PROXY=http://squid-proxy.mycompany.com:3128/
Trying to download a resource:
$ wget https://repository.apache.org/content/groups/snapshots/org/apache/karaf/apache-karaf/4.1.1-SNAPSHOT/apache-karaf-4.1.1-20170315.084054-35.tar.gz
The download hangs for 2 minutes until it gets started. A retry shows the same results, the download starts after 2 minutes showing:
--2017-03-16 09:31:26-- https://repository.apache.org/content/groups/snapshots/org/apache/karaf/apache-karaf/4.1.1-SNAPSHOT/apache-karaf-4.1.1-20170314.154157-34.tar.gz
Resolving squid-proxy.mycompany.com (squid-proxy.mycompany.com)... 10.152.132.41
Connecting to squid-proxy.mycompany.com (squid-proxy.mycompany.com)|10.152.132.41|:3128... connected.
cache.log:
2017/03/16 10:17:47 kid1| Shutdown: NTLM authentication.
2017/03/16 10:17:47 kid1| Shutdown: Negotiate authentication.
2017/03/16 10:17:47 kid1| Shutdown: Digest authentication.
2017/03/16 10:17:47 kid1| Shutdown: Basic authentication.
CPU Usage: 0.084 seconds = 0.052 user + 0.032 sys
Maximum Resident Size: 113840 KB
Page faults with physical i/o: 0
2017/03/16 10:17:48 kid1| Starting Squid Cache version 3.5.12 for x86_64-pc-linux-gnu...
2017/03/16 10:17:48 kid1| Service Name: squid
2017/03/16 10:17:48| pinger: Initialising ICMP pinger ...
2017/03/16 10:18:09.579 kid1| 44,2| peer_select.cc(258) peerSelectDnsPaths: Find IP destination for: http://proxy.materna.de:8080/squid-internal-dynamic/netdb' via proxy.materna.de
2017/03/16 10:18:09.579 kid1| 44,2| peer_select.cc(280) peerSelectDnsPaths: Found sources for 'http://proxy.materna.de:8080/squid-internal-dynamic/netdb'
2017/03/16 10:18:09.579 kid1| 44,2| peer_select.cc(281) peerSelectDnsPaths: always_direct = ALLOWED
2017/03/16 10:18:09.579 kid1| 44,2| peer_select.cc(282) peerSelectDnsPaths: never_direct = DUNNO
2017/03/16 10:18:09.579 kid1| 44,2| peer_select.cc(286) peerSelectDnsPaths: DIRECT = local=0.0.0.0 remote=139.2.1.3:8080 flags=1
2017/03/16 10:18:09.579 kid1| 44,2| peer_select.cc(295) peerSelectDnsPaths: timedout = 0
2017/03/16 10:18:12.279 kid1| 44,2| peer_select.cc(258) peerSelectDnsPaths: Find IP destination for: http://roxy.materna.de:8080/squid-internal-dynamic/netdb' via roxy.materna.de
2017/03/16 10:18:12.279 kid1| 44,2| peer_select.cc(280) peerSelectDnsPaths: Found sources for 'http://roxy.materna.de:8080/squid-internal-dynamic/netdb'
2017/03/16 10:18:12.279 kid1| 44,2| peer_select.cc(281) peerSelectDnsPaths: always_direct = ALLOWED
2017/03/16 10:18:12.279 kid1| 44,2| peer_select.cc(282) peerSelectDnsPaths: never_direct = DUNNO
2017/03/16 10:18:12.279 kid1| 44,2| peer_select.cc(286) peerSelectDnsPaths: DIRECT = local=0.0.0.0 remote=139.2.1.4:8080 flags=1
2017/03/16 10:18:12.279 kid1| 44,2| peer_select.cc(295) peerSelectDnsPaths: timedout = 0
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(258) peerSelectDnsPaths: Find IP destination for: repository.apache.org:443' via proxy.materna.de
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(258) peerSelectDnsPaths: Find IP destination for: repository.apache.org:443' via proxy.materna.de
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(258) peerSelectDnsPaths: Find IP destination for: repository.apache.org:443' via roxy.materna.de
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(258) peerSelectDnsPaths: Find IP destination for: repository.apache.org:443' via roxy.materna.de
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(280) peerSelectDnsPaths: Found sources for 'repository.apache.org:443'
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(281) peerSelectDnsPaths: always_direct = DENIED
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(282) peerSelectDnsPaths: never_direct = ALLOWED
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(292) peerSelectDnsPaths: cache_peer = local=0.0.0.0 remote=139.2.1.3:8080 flags=1
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(292) peerSelectDnsPaths: cache_peer = local=0.0.0.0 remote=139.2.1.3:8080 flags=1
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(292) peerSelectDnsPaths: cache_peer = local=0.0.0.0 remote=139.2.1.4:8080 flags=1
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(292) peerSelectDnsPaths: cache_peer = local=0.0.0.0 remote=139.2.1.4:8080 flags=1
2017/03/16 10:18:37.951 kid1| 44,2| peer_select.cc(295) peerSelectDnsPaths: timedout = 0
access.log
1489656077.628 159679 10.30.216.160 TCP_TUNNEL/200 26328966 CONNECT repository.apache.org:443 - ANY_OLD_PARENT/139.2.1.4 -
Any hints?
Jens
Gesendet: Donnerstag, 16. März 2017 um 09:27 Uhr
Von: "Amos Jeffries" <squid3 at treenet.co.nz>
An: squid-users at lists.squid-cache.org
Betreff: Re: [squid-users] No failover when default parent proxy fails (Squid 3.5.12)
On 16/03/2017 7:05 p.m., Jens Offenbach wrote:
> Thanks for your quick response...
>
> I have also configured, but the value seems not to be honored:
> connect_timeout 30 seconds
>
> The primary peer is down, but Squid does not print any "Dead parent"
> in the logs. Every HTTPS request is forwarded to the primary peer and
> it takes 1 minute until the secondary peer gets used, even with
> "connect_timeout 30 seconds". I think, I am facing the first issue
> that has been fixed by your patch.
>
The global config options being ignored completely is correct because
your peer have individual connect-timeout=5 settings.
So, those 5sec timeouts should be used instead now as before.
Though note that they apply only to how long a TCP connection (SYN,
SYN-ACK) is waited for. There is also a dns_timeout and peer selection
timeout that apply separately to the act of connecting. And a
forward_timeout global limit that all those operations have to fit
within, including retries.
Did you have a chance to try the debug setting I suggested at the
beginning? That will give you an immediate view about what Squid is
detecting as usable paths for each and every request and at what times
relative to the DEAD/LIVE notice.
> Are there any plans to backport this fix to Xenial APT repositories
> or to create a new Debian package for Squid4/5?
That is up to the Ubuntu server team, but I think it Unlikely. Zesty is
the current stable and things like this generally dont have enough
widespread impact to qualify for LTS backports.
Debian is now frozen to stabilize for the "Buster" release, that will
contain Squid-3.5.23 plus some few critical patches which are already
set. A Squid-4 package is ready and waiting for the release freeze to
end before it goes public in the Debian Unstable/Testing repos.
Amos
_______________________________________________
squid-users mailing list
squid-users at lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
More information about the squid-users
mailing list