[squid-users] squid crash: ERROR: system call failure while accepting a TLS connection

[Martin A. Brooks]

Hello Alex,

On 2024-11-18 18:03, Alex Rousskov wrote:
> On 2024-11-18 09:59, Martin A. Brooks wrote:
> 
>> I am running 3 squid instances behind a load balancer.  It was running 
>> fine for a couple of weeks but but there were suddenly tens of 
>> thousands of this sort of message in the log:
> 
>> squid[507015]: ERROR: system call failure while accepting a TLS 
>> connection on conn705596 local=[redacted]:3128 remote=[redacted]:34830 
>> FD 56 flags=1: 0xaaaae5711490*1
> 
> If you upgrade to a supported Squid version (v6+), the trailing hex 
> number will be replaced with potentially important error details. I do 
> not recall whether Squid handles these particular TLS errors correctly, 
> but modern Squids (v6+) have a higher chance of doing so.

I somewhat expected "try a newer squid" to be in the suggestions :)


> To avoid a misunderstanding, while the subject of this email thread 
> says "crash", the email describes a controlled/orderly/expected 
> emergency exit after a fatal error beyond Squid control.

Agreed, understood.



> 
> 
>> squid[507015]: logfileHandleWrite: daemon:/var/log/squid/access.log: 
>> error writing ((32) Broken pipe)
> 
> I suspect that something has killed (or prevented a restart of) helper 
> processes that are responsible for writing access.log entries. Failure 
> to log requests is currently fatal ("by design") for Squid, but the 
> true problem/trigger lies elsewhere -- why did the helper process die? 
> I recommend studying system logs for any unusual preceding 
> events/activity/reports. For _example_, it is possible that your system 
> ran out of some OS resources and/or is under DoS attack.


I had a good trawl through the logs, and happened to have some metrics 
running, and I haven't been able to find any suggestion that any of the 
systems ran out of resources.  Disk, memory and CPU all seem fine.

I'll rebuild the systems with a newer squid (probably Ubuntu LTS and 
their shipped package) and see if the problem reoccurs.


Thanks for looking.

Regards

Martin.