[squid-users] load balancing and site failover

[Amos Jeffries]

On 25/03/2015 9:55 a.m., brendan kearney wrote:
> Was not sure if bugzilla was used for mailing list issues.  If you would
> like me to open one, I will but it looks like the list is working again.

Bugzilla is used, list bugs under the "project services" product.


As for your query...

> On Mar 24, 2015 2:25 PM, "Brendan Kearney" wrote:
> 
>> On Tue, 2015-03-24 at 10:18 -0400, Brendan Kearney wrote:
>>> while load balancing is not a requirement in a proxy environment, it
>>> does afford a great deal of functionality, scaling and fault tolerance
>>> in one.  several if not many on this list probably employ them for their
>>> proxies and likely other technologies, but they are not all created
>>> equal.
>>>
>>> i recently looked to see if a specific feature was in HAProxy.  i was
>>> looking to see if HAProxy could reply to a new connection with a RST
>>> packet if no pool member was available.
>>>
>>> the idea behind this is, if all of the proxies are not passing the
>>> service check and are marked down by the load balancer, the reply of a
>>> RST in the TCP handshake (i.e. SYN -> RST, not SYN -> SYN/ACK -> ACK)
>>> tells the browser to failover to the next proxy assigned by the PAC
>>> file.
>>>
>>> where i work, we have this configuration working.  the load balancers
>>> are configured with the option to send a reset when no proxy is
>>> available in the pool.  the PAC file assigns all 4 of the proxy VIPs in
>>> a specific order based on which proxy VIP is assigned as the primary.
>>> In every case, if the primary VIP does not have an available pool
>>> member, the browser fails over to the next in the list.  failover would
>>> happen again, if the secondary VIP replies with a RST during the
>>> connection establishing.  the process repeats until a TCP connection
>>> establishes or all proxies assigned have been exhausted.  the browser
>>> will use the proxy VIP that it successfully connects to, for the
>>> duration of the session.  once the browser is closed and reopened, the
>>> evaluation of the PAC file occurs again, and the process starts anew.
>>> plug-ins such as Proxy Selector are the exception to this, and can be
>>> used to reevaluate a PAC file by selecting it for use.
>>>
>>> we have used this configuration several times, when we found an ISP link
>>> was flapping or some other issue more global in nature than just the
>>> proxies was affecting our egress and internet access.  i can attest to
>>> the solution as working and elegantly handling site wide failures.
>>>
>>> being that the solutions where i work are proprietary commercial
>>> products, i wanted to find an open source product that does this.  i
>>> have been a long time user of HAProxy, and have recommended it for
>>> others here, but sadly they cannot perform this function.  per their
>>> mailing list, they use the network stack of the OS for connection
>>> establishment and cannot cause a RST to be sent to the client during a
>>> TCP handshake if no pool member is available.
>>>
>>> they suggested an external helper that manipulates IPTables rules based
>>> on a pool member being available.  they do not feel that a feature like
>>> this belongs in a layer 4/7 reverse proxy application.

They are right. HTTP != TCP.

In particular TCP depends on routers having a full routing map of the
entire Internet (provided by BGP) and deciding the best upstream hop
based on that global info. Clients have one (and only one) upstream
router for each server they want to connect to.

In HTTP each proxy (aka router) performs independent upstream connection
attempts, failover, and verifies it worked before responding to the
client with a final response. Each proxy only has enough detail to check
its upstream(s). Each proxy can connect to any server (subject to ACLs).


>>>
>>> my search for a load balancer solution went through ipvsadm, balance and
>>> haproxy before i selected haproxy.  haproxy was more feature rich than
>>> balance, and easier to implement than ipvsadm.  do any other list
>>> members have a need for such a feature from their load balancers?  do
>>> any other list members have site failover solutions that have been
>>> tested or used and would consider sharing their design and/or pain
>>> points?  i am not looking for secret sauce or confidential info, but
>>> more high level architecture decisions and such.


I havent tested it but this should do what you are asking:

 acl err http_status 500-505 408
 deny_info TCP_RESET err
 http_reply_access deny err

It replaces the response from Squid with a TCP RST packet.

Amos