[squid-users] Squid TCP_TUNNEL_ABORTED/200

[Amos Jeffries]

On 4/05/24 02:29, Emre Oksum wrote:
> Hi everyone,
> 
> I'm having a issue with Squid Cache 4.10 which I cannot fix for weeks 
> now and kinda lost at the moment. I will be appreciated if someone can 
> guide me through the issue I'm having.
> I need to create a IPv6 HTTP proxy which should match the entry address 
> to outgoing TCP address. For example, if user is connecting from 
> fe80:abcd::1 it should exit the HTTP proxy from the same address. We got 
> like 50k addresses like this at the moment.

What your "for example,..." describes is Transparent Proxy (TPROXY).


However, what you have in the config below is very different. The IP the 
client is connected **to** (not "from") is being pinned on outgoing 
connections.


> The issue is, client connecting to the proxy is receiving "EOF" or 
> "FLOW_CONTROL_ERROR" on their side.

The FLOW_CONTROL_ERROR is not something produced by Squid. Likely it 
comes from the TCP stack and/or OS routing system.

The EOF may be coming from either Squid or the OS. It also may be 
perfectly normal for the circumstances, or a side effect of an error 
elsewhere.


To solve will require identifying exactly what is sending those signals, 
and why. Since they are signals going to the client, focus on the 
client->Squid connections (not the Squid->server ones you talk about 
testing below).



> When I test connection by connecting 
> to whatismyip.com <http://whatismyip.com> everything works fine and 
> entry IP always matches with outgoing IP for each of the 50k addresses. 
> Client tells me this problem occurs both at GET and POST requests with 
> around 10 MB of data.

Well, you are trying to manually force certain flow patterns that 
prohibit or break some major HTTP performance features. Some problems 
are to be expected.

The issues which I expect to occur in your proxy would not show up in a 
trivial outgoing-IP or connectivity test.


> I initially thought that could be related to server resources being 
> drained but upon inspecting server resource usage, Squid isn't even 
> topping at 100% CPU or RAM anytime so not that.
> 

IMO, "FLOW_CONTROL_ERROR" is likely related to quantity of traffic 
flooding through the proxy to specific origin servers.

The concept you are implementing of the outgoing TCP connection having 
the same IP as the incoming connection reduces the available TCP sockets 
by 25%. Prohibiting the OS from allocating ports on otherwise unused 
outgoing addresses when



> My Squid.conf is like this at the moment:

Some improvements highlighted inline below.
Nothing stands out to me as being related to your issues.

> 
> auth_param basic program /usr/lib/squid/basic_ncsa_auth /etc/squid/passwd
> acl auth_users proxy_auth REQUIRED
> http_access allow auth_users
> http_access deny !auth_users

Above two lines are backwards. Deny first, then allow.


> cache deny all
> dns_nameservers <nameservers here>
> dns_v4_first off
> via off
> forwarded_for delete
> follow_x_forwarded_for deny all
> server_persistent_connections off

*If* the issue turns out to be congestion on Squid->server connections
enabling this might be worthwhile. Otherwise it should be fine.


> max_filedesc 1048576

You can remove that line. "max_filedesc" was a RedHat hack from 20+ 
years ago when the feature was experimental.

Any value you set on the line above, will be erased and replaced by the 
line below:


> max_filedescriptors 1048576
> workers 8
> http_port [::0]:1182

Above is just a complicated way to write:

  http_port 1182


Any particular reason not to use the registered port 3128 ?
(Not important, just wondering.)


> acl binding1 myip fe80:abcd::1
> tcp_outgoing_address fe80:abcd::1 binding1
> acl binding2 myip fe80:abcd::2
> tcp_outgoing_address fe80:abcd::2 binding2
> acl binding3 myip fe80:abcd::3
> tcp_outgoing_address fe80:abcd::3 binding3
> ...
> ...
> ...
> access_log /var/log/squid/access.log squid

> cache_store_log none

You can erase this line.
This is default setting. No need to manually set it.


> cache deny all

You can erase this line.
This "cache deny all" exists earlier in the config.


> 
> I've tried to get a PCAP file and realized when client tries to connect 
> with a new IPv6 address, Squid is not trying to open a new connection 
> instead tries to resume a previously opened one on a different outgoing 
> IPv6 address.

Can you provide the trace demonstrating that issue?

Although, as noted earlier your problems are apparently on the client 
connections. This is about server connections behaviour.


> I set server_persistent_connections off which should have 
> disabled this behavior but it's still the same.

Nod. Yes that should forbid re-use of connections.

I/we will need to see the PCAP trace along with a cache.log generated 
using "debug_options ALL,6" to confirm a bug or identify other breakage 
though.



> I tried using a newer 
> version of Squid but it behaved differently and did not follow my 
> outgoing address specifications and kept connecting on IPv4.

That would seem to indicate that your IPv4 connectivity is better than 
your IPv6 connectivity. Later Squid use various "Happy Eyeballs" 
implementations for the server selection.

You can usually work around this by configuring the DNS server specified 
by dns_nameservers to only deliver IPv6 results when a mixed set are 
available.


HTH
Amos