[squid-users] Squid with SMP, CARP and a forwarding loop

[Amos Jeffries]

On 30/10/2015 1:45 p.m., Mike.Hodgkinson wrote:
> I have been attempting to setup a squid forward proxy with one frontend 
> and two backends as per configuration example 
> http://wiki.squid-cache.org/ConfigExamples/SmpCarpCluster
> 
> My issue is that only the first attempt comes from the cache and then 
> additional requests are downloaded direct by the frontend instead of from 
> the backend caches. I suspect it is due to a detected forwarding loop 
> which shows up in the logs:
> 
> 2015/10/30 13:07:49.239 kid1| 44,3| peer_select.cc(137) peerSelect: 
> e:=XIWV/0x7f7bfee2e730*2 http://127.0.0.1:40
> 02/squid-internal-dynamic/netdb
> 2015/10/30 13:07:49.239 kid1| 20,3| store.cc(466) lock: peerSelect locked 
> key 64AAA11C8DEF57153B10BA2C9D2F3D60 e:=XIWV/0x7f7bfee2e730*3
> 2015/10/30 13:07:49.240 kid1| 44,3| peer_select.cc(441) peerSelectFoo: GET 
> 127.0.0.1
> 2015/10/30 13:07:49.240 kid1| 44,3| peer_select.cc(468) peerSelectFoo: 
> peerSelectFoo: direct = DIRECT_YES (forwarding loop detected)
> 2015/10/30 13:07:49.240 kid1| 44,3| peer_select.cc(477) peerSelectFoo: 
> peerSelectFoo: direct = DIRECT_YES
> 2015/10/30 13:07:49.240 kid1| 44,2| peer_select.cc(258) 
> peerSelectDnsPaths: Find IP destination for: 
> http://127.0.0.1:4002/squid-internal-dynamic/netdb' via 127.0.0.1
> 
> I can force the backend caches to be used successfully with this option 
> "never_direct allow all" however I would like to resolve the underlying 
> issue.

The above is an internal Squid request from the fontend to its backends.
This one specifically is going direct due to a hack in the code.

You can avoid it by adding "no-netdb-exchange" to the cache_peer lines.
I'm not sure if that will affect the CARP selection since these requests
are one of the types feeding into the peer up/down/overloaded
monitoring. With "no-query" also in use you will be left with the client
HTTP traffic being the only source of that data which carp depends on.

Or using that "never_direct allow all" will override that code hack.

> 
> I have no iptables configured on this server and have made sure the 
> environment variable http_proxy is not set. Also I have tested this on 
> Squid 3.4.8 and 3.5.10 on Debian.

Since you have no iptables rules configured the traffic arriving in port
3129 will be completely borked.

Either, remove that port 3129 line from the frontend config and use port
3128 for testing until you are ready to setup TPROXY properly;

Or, setup the TPROXY iptables and routing rules and test the proxy
exactly as the clients would be using it.


> 
> My config is below:
> #/etc/squid/squid.conf#
> debug_options = ALL,3
> cachemgr_passwd ******

NOTE: if that was your actual password you now need to change it.

> acl localnet src 10.1.0.0/16
> acl localnet src 10.2.0.0/16
> acl localnet src 192.168.0.0/23
> acl localnet src fe80::/10
> acl squid_servers src 10.1.209.0/24

 See below...

<snip>
> #/etc/squid/squid-frontend.conf#
> http_port 3128
> http_port 3129 tproxy
> http_access allow localhost manager
> http_access deny manager
> http_access allow localhost
> http_access allow localnet
> http_access allow squid_servers

With squid_servers IP range being entirely within "localnet" this
"http_access allow squid_servers" line is not doing anything.
You can simplify by removing it.


> htcp_access allow squid_servers
> htcp_access deny all
> cache_peer 127.0.0.1 parent 4002 0 carp login=PASS name=backend-kid2 
> no-query
> cache_peer 127.0.0.1 parent 4003 0 carp login=PASS name=backend-kid3 
> no-query
> prefer_direct off
> nonhierarchical_direct off
> memory_replacement_policy heap LRU
> cache_mem 2048 MB
> access_log /var/log/squid3/frontend.access.log
> cache_log /var/log/squid3/frontend.cache.log
> visible_hostname frontend.cloud.solnet.nz
> 
> #/etc/squid/squid-backend.conf#
> http_port 127.0.01:400${process_number}
> http_access allow localhost
> cache_mem 5 MB
> cache_replacement_policy heap LFUDA
> maximum_object_size 1 GB
> cache_dir rock /cache/rock 20480 max-size=32768
> cache_dir aufs /cache/${process_number} 20480 128 128 min-size=32769
> visible_hostname backend${process_number}.cloud.solnet.nz
> access_log /var/log/squid3/backend${process_number}.access.log
> cache_log /var/log/squid3/backend${process_number}.cache.log
> 
> I did have visible_hostname set to backend.cloud.solnet.nz but that did 
> not help either.

Nod. All that did previously was prevent forwarding loops between the
backends in case something unusually nasty happened in the
routing/iptabels layers. What you have now is okay and slightly better
for diagnosing the issues.


Amos