[squid-users] Happy Eyeballs and "connect_timeout" in squid 3.4.12

[Amos Jeffries]

On 30/04/2015 7:05 a.m., Tom Tom wrote:
> Thank you Amos, for this explanation.
> 
> 
> On Wed, Apr 29, 2015 at 3:02 PM, Amos Jeffries wrote:
>> On 29/04/2015 7:38 p.m., Tom Tom wrote:
>>> Hi
>>>
>>> I'm running squid (3.4.12) on a IPv6/IPv4-dual-stack system.
>>>
>>> While accessing the test-site "http://test.rx.td.h.labs.apnic.net", I
>>> encountered a 60s connection-timeout (configurable with
>>> connect_timeout) while squid is making 5 IPv6-connection-attempts
>>> (SYN), before it tries to connect with IPv4 (which is working on the
>>> test-site). I can decrease the "connect_timeout"-value to 1 second.
>>> This behaves in a better "surf"-experience and results in a 1s-timeout
>>> (also only 1 IPv6-SYN) instead of the default 60s timeout.
>>>
>>> Why does squid not tries to connect first IPv6 (based on the host's
>>> address preference-policy) and then - in case of a failure - switch to
>>> IPv4 during a 300ms timeout (like current Browsers are doing)?
>>
>> Several reasons:
>>
>> 1) The default builds and installs do try IPv6 first in accordance with
>> RFC 6540. Check your config for a "dns_v4_first" directive which forces
>> IPv4 to be tried first.
> 
> According to RFC 6555:  "Over time, as most content is available via
> IPv6, the amount of IPv4 traffic will decrease.". With forcing this
> directive, I reduce the chance for connecting with IPv6 and my
> outbound connections are probably a long time with IPv4. This is maybe
> not the behaviour we want?

If you want to encourage IPv6 usage let Squid operate at its default
behaviour and fix the issues in the network which make any given request
go particularly worse than in IPv4. Sadly many of these are caused by
external sysadmins choices nowdays either to run with outdated machinery
or to explicitly break IPv6 in the name of disabling it.

As a proponent of IPv6 adoption myself I have written the IPv6
behaviours into Squid to prefer IPv6 over IPv4 whenever possible. Long
before RFC 6540 required it.


>>
>> 2) Squid is not the OS built-in resolver. Any obeying of that policy by
>> Squid is purely arbitrary. The host systems DNS resolver policy does not
>> supposed to affect standalone resolvers such as Squids internal one.
>> Particularly when there are squid.conf directives overriding the
>> resolv.conf behaviour (eg. dns_nameservers).
>>
>> dns_v4_first was a partial implementation added for
>> <http://bugs.squid-cache.org/show_bug.cgi?id=3086>.
>>
>>
>> 3) when performed by middleware such as Squid the "Happy Eyeballs"
>> algorithm is heavily destructive.
>>
>> A browser is consuming at minimum 2 network sockets to perform "Happy
>> Eyeballs".
>>
>> At the middlware each of those translates to potentially 3 sockets
>> (total 6 proxy sockets, 2 outgoing server sockets). If the middleware
>> were to perform "Happy Eyeballs" itself that would increase to 4 sockets
>> (total 8 proxy sockets, 4 outgoing server sockets).
> 
> But only in the parallel way (1 x IPv6 and 1 x IPv4)?

No, multiplexed. Each hop has both IPv4 and IPv6 outbound possibilities
for each individual packet regardless of the inbound type. "Happy
Eyeballs" worst-case is a straight exponential 2^N socket usage at the
server where N is proxy hop distance from client. Best-case occurs when
an admin chooses to disable IPv4 or IPv6 and cuts the exponential growth
from their hop in half.

So the algorithm happening in middleware would actively encourage bad
network practices by sysadmin. Grr :-(

> 
> Tests with the current curl (I know, curl != squid) behaves not in
> doing two similar (parallel) TCP-Connections (1x SYN for IPv6 and 1x
> SYN for IPv4). Instead, curl tries IPv6 first and in case of an
> connection-error, it tries after a few milliseconds with IPv4. This
> way, not a lot of sockets should be consumed. Does this not behave
> like a native IPv4-Stack? Squid would behave like curl, if I would be
> able to change the connect_timeout to milliseconds.

That sequential operation is already the current behaviour of Squid.
Just with resolution of seconds on the timeout and configurable choice
between ordering of {IPv6,IPv4} or {IPv4,IPv6} as to what gets tried first.

The browser "Happy Eyeballs" algorithm you were talking about /
proposing works quite differenty. All A and AAAA record have DNS queries
generated at once - as those replies happen all listed IPs have TCP SYN
packets generated at once and all of them are sent. It ends when one TCP
SYN packet gets a success response. End of spec.

That performance numbers that come out of it are great ... for a single
point-to-point connection.
It looks less wonderful and more like a DoS attack in all other cases,
especially when taking server resource side-effects into account.

> 
> Is there a well-known restriction (which I don't now, actually), by
> setting the connect_timeout to 1 second for all those IPv6-Adresses,
> which aren't connectable and for which the IPv4-Stack should be used
> after 1s timeout? Is this a practicable way?

There are two restrictions:

 1) the OS unix time (time_t) has a minimum resolution of 1 second.
Anything using more detailed time resolution is paying (in mutiple ms of
extra latency) for expensive syscalls to find out what the nanosecond
clocks in the system are.

 2) the event loop checking for what timeouts have occured cycles once
per second. The guarantee provided by Squid is that a timeout action
will have *at least* the requested N seconds before it triggers.

Also, to a lesser degree translating between timescales for this
directive with all the other timouts it interacts with would be a
performance drag and annoying to code.

Amos