[squid-users] squid hangs and dies and can not be killed - needs system reboot

[Amos Jeffries]

On 19/12/23 16:29, Amish wrote:
> Hi Alex,
> 
> Thank you for replying.
> 
> On 19/12/23 01:14, Alex Rousskov wrote:
>> On 2023-12-18 09:35, Amish wrote:
>>
>>> I use Arch Linux and today I updated squid from squid 5.7 to squid 6.6.
>>
>> > Dec 18 13:01:24 mumbai squid[604]: kick abandoning conn199
>>
>> I do not know whether the above problem is the primary problem in your 
>> setup, but it is a red flag. Transactions on the same connection may 
>> get stuck after that message; it is essentially a Squid bug.
>>
>> I am not sure at all, but this bug might be related to Bug 5187 
>> workaround that went into Squid v6.2 (commit c44cfe7): 
>> https://bugs.squid-cache.org/show_bug.cgi?id=5187
>>
>> Does Squid accept new TCP connections after it enters what you 
>> describe as a dead state? For example, does "telnet 127.0.0.1 8080" 
>> establishes a connection if executed on the same machine as Squid?
>>
> Yes it establishes connection. But I do not know what to do next. 
> Browser showed "Connection timed out" message. But I believe browser's 
> also connected but nothing happened afterwards.
> 

Ah ... that port is an interception port. It should *not* connect.

Please ensure your firewall contains the "-t mangle" rules for each 
interception port you use. As shown at 
<https://wiki.squid-cache.org/ConfigExamples/Intercept/LinuxDnat>


>>
>> > kill -9 does nothing
>>
>> Is it possible that you are trying to kill the wrong process? You 
>> should be killing this process AFAICT:
>>
>> > root         601  0.0  0.2  73816 22528 ?        Ss   12:59 0:02
>> > /usr/bin/squid -f /etc/squid/btnet/squid.btnet.conf --foreground -sYC
> 
> I did not clarify but all processes needed SIGKILL and vanished except 
> the Dead squid process which still remained.
> 
> # systemctl stop squid
> 
> Dec 19 08:46:38 mumbai systemd[1]: squid.service: State 'stop-sigterm' 
> timed out. Killing.


FWIW, Squid default shutdown grace period for clients to disconnect is 
longer that systemd typically is willing to wait for a service shutdown.

Please set "shutdown_lifetime 10 seconds" in your squid.conf.


> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 601 
> (squid) with signal SIGKILL.
> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 604 
> (squid) with signal SIGKILL.

This is systemd running the command " kill -9 604 ".

Per the Squid code: "XXX: In SMP mode, uncatchable SIGKILL only kills 
the master process".

You can try SIGTERM instead, and repeat up to 3 times if the first does 
not close the process.



HTH
Amos