[squid-users] IPv4 addresses go missing - markAsBad wrong?

[Alex Rousskov]

On 2024-01-08 08:31, Stephen Borrill wrote:
> I'm trying to determine why squid 6.x (seen with 6.5) connected via 
> IPv4-only periodically fails to connect to the destination and then 
> requires a restart to fix it (reload is not sufficient).
> 
> The problem appears to be that a host that has one address each of IPv4 
> and IPv6 occasionally has its IPv4 address go missing as a destination. 
> On closer inspection, this appears to happen when the IPv6 address (not 
> the IPv4) address is marked as bad. A log fragment is as follows:
> 
> 2024/01/08 13:18:39.974 kid1| 44,2| peer_select.cc(460) resolveSelected: 
> Find IP destination for: clientservices.googleapis.com:443' via 
> clientservices.googleapis.com
> 2024/01/08 13:18:39.974 kid1| 44,2| peer_select.cc(1174) handlePath: 
> PeerSelector82284 found conn696198 local=0.0.0.0 
> remote=142.250.187.227:443 HIER_DIRECT flags=1, destination #1 for 
> clientservices.googleapis.com:443
> 2024/01/08 13:18:39.974 kid1| 44,2| peer_select.cc(1174) handlePath: 
> PeerSelector82284 found conn696199 local=[::] 
> remote=[2a00:1450:4009:820::2003]:443 HIER_DIRECT flags=1, destination 
> #2 for clientservices.googleapis.com:443
> 2024/01/08 13:18:39.974 kid1| 44,2| peer_select.cc(479) resolveSelected: 
> PeerSelector82284 found all 2 destinations for 
> clientservices.googleapis.com:443
> 2024/01/08 13:18:40.245 kid1| 14,2| ipcache.cc(1031) markAsBad: 
> [2a00:1450:4009:820::2003]:443 of clientservices.googleapis.com
> 2024/01/08 13:18:40.245 kid1| 14,3| ipcache.cc(946) seekNewGood: 
> succeeded for clientservices.googleapis.com: [2a00:1450:4009:820::2003] 
> #2/2-1
> 2024/01/08 13:18:40.245 kid1| 14,3| ipcache.cc(978) restoreGoodness: 
> cleared all IPs for clientservices.googleapis.com; now back to 
> [2a00:1450:4009:820::2003] #2/2-1
> 2024/01/08 13:18:42.065 kid1| 14,3| Address.cc(389) lookupHostIP: Given 
> Non-IP 'clientservices.googleapis.com': hostname or servname not 
> provided or not known
> 2024/01/08 13:18:42.065 kid1| 44,2| peer_select.cc(460) resolveSelected: 
> Find IP destination for: clientservices.googleapis.com:443' via 
> clientservices.googleapis.com
> 2024/01/08 13:18:42.065 kid1| 14,3| Address.cc(389) lookupHostIP: Given 
> Non-IP 'clientservices.googleapis.com': hostname or servname not 
> provided or not known
> 2024/01/08 13:18:42.065 kid1| 44,2| peer_select.cc(1174) handlePath: 
> PeerSelector82372 found conn697148 local=[::] 
> remote=[2a00:1450:4009:820::2003]:443 HIER_DIRECT flags=1, destination 
> #1 for clientservices.googleapis.com:443
> 2024/01/08 13:18:42.065 kid1| 44,2| peer_select.cc(479) resolveSelected: 
> PeerSelector82372 found all 1 destinations for 
> clientservices.googleapis.com:443
> 
> 
> This shows two subsequent connection attempts to 
> clientservices.googleapis.com. The first one has both IPv4 and IPv6 
> destinations. The IPv6 address is passed to markAsBad. 

Yes.


> After that the IPv4 address is not listed as a destination.

I do not see that. I see IPv6 address being selected as the first 
destination (instead of the IPv4 address).

I cannot explain why that happens though. Moreover, a combination of 
certain lines in your debug output near "seekNewGood" do not make sense 
to me -- I do not see how it is possible for Squid to display those 
exact debugging details, but I am probably missing something. Can you 
retest and repost similar lines with 14,9 (or at least 14,7) added to 
your debug_options (or share those lines privately; the more lines you 
can share, the better)?


> Note that there have been many connections to 
> clientservices.googleapis.com prior to this where markAsBad was not 
> called, even though IPv6 connectivity was never available.

No markAsBad() is probably normal if Squid did not try to establish an 
IPv6 connection or did not wait long enough to know the result of that 
attempt. However, that does not explain why Squid selected an IPv6 
address as the next "good" address right after marking that IPv6 address 
as bad (at "restoreGoodness" line) when there was another good IP 
address available. It is as if Squid stored two identical IPv6 addresses 
(and not IPv4 ones), but that should not happen either.

Alex.