[squid-users] Reloading squid service results in connection resets

[Toler, Matt]

Hello!

I hope you all are well. We've run into a troublesome issue and hope to get some guidance.

We have an automated workflow that will reload the squid configuration if any changes are made. In our use case the changes are dynamic and happen often. We were able to determine that frequency isn't the problem as the issue can be manually reproduced in the absents of any automation or significant load.

Environment:
OS: RHEL 7.9
Kernel: 3.10.0-1160.62.1.el7.x86_64
Squid: 4.14

In our test case we're using the AWS CLI to get information from EC2 instances. While looping "ec2 describe-instances" through the squid via a load balancer everything works great until we reload the service. Most times the command will pause and return the requested output after a few more seconds. However, more times than is tolerable when the service is reloading the command will return error "Failed to connect to proxy URL" to the client. With the AWS CLI debug enabled before the "Failed to connect to proxy URL" is thrown we see "ConnectionResetError: [Errno 104] Connection reset by peer". We then ran packet captures and were able to determine that the reset was coming from the server running squid at the time the service was reloading. This connection is not logged by squid at all which was confusing to us that we had any client connection issues as the squid logs are clean. We are reloading the service with systemd but were able to reproduce the issue with "squid -k reconfigure" as well.

Our current plan is to upgrade our servers to RHEL8 and compile squid 5.6 in hopes that this issue will go away but we may need to go so far as programmatically removing a given squid node from the Load balancer before any service reload.

That said, it was our understanding that reloading the service would not disturb any old connections and new connections would receive the new configuration. We were wondering if anyone else has encountered any issues with connection resets of client traffic during squid service reload? Or may otherwise have any thoughts on this issue. Please let me know if I can provide any further detail.

Thanks in advance.

Regards,
Matt







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20220613/0c472565/attachment.htm>