[squid-users] ERR_CONNECT_FAIL 110

[Amos Jeffries]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 21/12/2014 10:12 a.m., Alfredo Rezinovsky wrote:
> El 19/12/14 a las 12:53, Amos Jeffries escibió: On 20/12/2014 4:21
> a.m., Alfredo Rezinovsky wrote:
>>>> I have a few TPROXY implementations with squid. In only one
>>>> of them recently I'm getting lots of: "x-squid-error:
>>>> ERR_CONNECT_FAIL 110" and some 504 timeouts.
>>>> 
>>>> Squid Cache: Version 3.4.10-20141218-r13197 configure
>>>> options: '--prefix=/opt/sepia/squid'
>>>> '--sysconfdir=/var/lib/sepia/' '--disable-auth'
>>>> '--disable-auto-locale' '--disable-cache-digests' 
>>>> '--disable-cpu-profiling' '--disable-debug-cbdata' 
>>>> '--disable-delay-pools' '--disable-devpoll' '--disable-ecap' 
>>>> '--disable-esi' '--disable-eui'
>>>> '--disable-external-acl-helpers' 
>>>> '--disable-follow-x-forwarded-for' '--disable-forw-via-db' 
>>>> '--enable-gnuregex' '--disable-htcp' '--disable-icap-client' 
>>>> '--disable-ident-lookups' '--enable-internal-dns' 
>>>> '--disable-ipf-transparent' '--disable-ipfw-transparent' 
>>>> '--disable-ipv6' '--disable-leakfinder'
>>>> '--disable-pf-transparent' '--disable-poll'
>>>> '--disable-select' '--disable-snmp' '--enable-ssl' 
>>>> '--disable-stacktraces' '--disable-translation' 
>>>> '--disable-url-rewrite-helpers' '--disable-wccp'
>>>> '--disable-wccpv2' '--disable-win32-service'
>>>> '--disable-x-accelerator-vary' '--disable-icmp'
>>>> '--disable-storeid-rewrite-helpers' '--enable-async-io'
>>>> '--enable-disk-io' '--enable-epoll' 
>>>> '--enable-http-violations' '--enable-inline' 
>>>> '--enable-kill-parent-hack' '--enable-linux-netfilter' 
>>>> '--enable-log-daemon-helpers' '--enable-removal-policies' 
>>>> '--enable-storeio' '--enable-unlinkd' 
>>>> '--enable-x-accelerator-vary' '--enable-zph-qos' 
>>>> '--with-default-user=nobody' '--with-logdir=/var/log/sepia' 
>>>> '--with-pthreads' '--with-included-ltdl' 
>>>> '--with-pidfile=/var/lib/sepia/squid.pid' 
>>>> '--with-netfilter-conntrack' --enable-ltdl-convenience
>>>> 
>>>> Is a custom compiled squid with everything I don't need
>>>> disabled.
>>>> 
>>>> Running in Ubuntu with kernel 3.13.0
>>>> 
>>>> PMTU from the proxy to both the servers and the clients seems
>>>> to be 1500.
>>>> 
>>>> Any clue?
> Nope you omitted the best clues. :-)
> 
> The access.log entries matching those errors would be a good start
> if you can identify them.
> 
> Amos
>> _______________________________________________ squid-users
>> mailing list squid-users at lists.squid-cache.org 
>> http://lists.squid-cache.org/listinfo/squid-users
> Shame on me
> 
> 1419108172.470  29936 172.16.1.2 TCP_MISS_ABORTED/000 0 GET 
> http://www.ibm.com/ - ORIGINAL_DST/172.233.13.247 - 1419108202.446
> 29971 172.16.1.2 TCP_MISS_ABORTED/000 0 GET http://www.ibm.com/ -
> ORIGINAL_DST/172.233.13.247 - 1419108212.325  30029 172.16.1.2
> TCP_MISS_ABORTED/000 0 GET http://www.ibm.com/ -
> ORIGINAL_DST/172.233.13.247 - 1419108232.487  30029 172.16.1.2
> TCP_MISS_ABORTED/000 0 GET http://www.ibm.com/ -
> ORIGINAL_DST/172.233.13.247 - 1419108262.453  29814 172.16.1.2
> TCP_MISS_ABORTED/000 0 GET http://www.ibm.com/ -
> ORIGINAL_DST/172.233.13.247 - 1419108294.101  59408 172.16.1.2
> TCP_MISS/503 469 GET http://xml.weather.yahoo.com/forecastrss? -
> ORIGINAL_DST/206.190.43.214 text/html 1419108295.670  60800
> 172.16.1.2 TCP_MISS/503 469 GET 
> http://download.finance.yahoo.com/d/333.txt? - 
> ORIGINAL_DST/209.191.96.200 text/html
> 
> All 503 errors are around 60 seconds. The same requests works whe
> the tproxy is not enabled.
> 

Okay, this says that you are intercepting the traffic. Squid tried
opening a connection to the same IP the client was connecting to.
(should work right?). But the TCP SYN packets sent out by Squid never
got any response.

Sometimes (ABORTED/000) the client gave up waiting and disconnected
after ~30sec.

Sometimes (MISS/503) Squid was the one to give up after ~60sec.

Since it is the outbound TCP connections from Squid that are dying.
Check the usual suspects:

 ICMP blocking  - only a very small sub-set of a few codes are
dangerous and need blocking, the rest are useful or mandatory for
reliable connectivity.

 path-MTU discovery - can be broken by ICMP packets being dropped or
bad MSS values on a tunnel/VPN interface,

 ECN and TCP Window Scaling - can be corrupted by old broken software
on the transit path,

 NAT on the outbound connections - can send packets to weird places.

Amos
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (MingW32)

iQEcBAEBAgAGBQJUlj8bAAoJELJo5wb/XPRjGx8H/2uyWG+PKh06b/aS1Mv5xbV8
M1p09RTLJ1gD4F4aasAQuHQyCqPI3VpyoURskr8hJWtpQjpE7dxvEMCP9fIlp7rX
ButRCUGtEOoZ1rvqNkSQKvTaWk2tO7kPg0/GDFO5k0f8s6zVDTfGbHFefSakjXm6
vPHamIBHcgVqlgC3JCqcRMgrLyZoBEyMhgCP9O4P7677TPyKKn7YeJVFquSwJ0do
8xJOsWnWd15W1waRyaHJLzn6wcv+DSJLl8NBDJF3WZqlt2Itnu/flQ2OJIdmEbXS
eB7b2oT53hf9QHeC3FpfozFuLvnj8WmsorQtvmO1rQSCY7kONH94Sk407+j+Wes=
=0UIE
-----END PGP SIGNATURE-----