[squid-users] squid hangs and dies and can not be killed - needs system reboot
Amish
anon.amish at gmail.com
Thu Dec 21 12:10:40 UTC 2023
Hi Alex,
Sorry for late reply. But this is a production system and hence
debugging is tough. May take few days.
However I noticed that everytime squid hangs (goes dead), the dmesg has
these errors.
[63567.802188] divide error: 0000 [#1] PREEMPT SMP PTI
[63567.802215] CPU: 3 PID: 617 Comm: squid Not tainted 6.6.7-arch1-1 #1
4505c4baa0b3d7c4037b0e8f5402626fa360717f
[63567.802238] Hardware name: Gigabyte Technology Co., Ltd.
H81M-S/H81M-S, BIOS F2 08/19/2015
[63567.802255] RIP: 0010:tcp_rcv_space_adjust+0xbe/0x160
[63567.802267] Code: f8 41 89 d0 29 d0 31 d2 49 0f af c2 49 f7 f0 45 8b
81 bc 04 00 00 44 0f b6 8b 10 06 00 00 31 d2 49 8d 04 42 48 98 48 c1 e0
08 <49> f7 f1 49 63 d0
48 98 48 39 d0 48 0f 47 c2 39 83 18 01 00 00 7c
[63567.802287] RSP: 0018:ffffba9f00da3c18 EFLAGS: 00010206
[63567.802296] RAX: 0000000004fb2800 RBX: ffff9e124acc1c80 RCX:
0000000eccd9dace
[63567.802304] RDX: 0000000000000000 RSI: 0000000037fa4ae8 RDI:
0000000000008349
[63567.802314] RBP: ffff9e124acc1c80 R08: 0000000000600000 R09:
0000000000000000
[63567.802322] R10: 00000000000161d2 R11: ffff9e1346621700 R12:
0000000000000000
[63567.802331] R13: ffff9e124acc1d58 R14: 0000000000000000 R15:
ffff9e124acc2214
[63567.802340] FS: 00007fc3d627c840(0000) GS:ffff9e1457380000(0000)
knlGS:0000000000000000
[63567.802350] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[63567.802358] CR2: 0000560b7e86d690 CR3: 0000000102f44001 CR4:
00000000001706e0
[63567.802367] Call Trace:
[63567.802372] <TASK>
[63567.802378] ? die+0x36/0x90
[63567.802386] ? do_trap+0xda/0x100
[63567.802392] ? tcp_rcv_space_adjust+0xbe/0x160
[63567.802401] ? do_error_trap+0x6a/0x90
[63567.802408] ? tcp_rcv_space_adjust+0xbe/0x160
[63567.802415] ? exc_divide_error+0x38/0x50
[63567.802423] ? tcp_rcv_space_adjust+0xbe/0x160
[63567.802431] ? asm_exc_divide_error+0x1a/0x20
[63567.802441] ? tcp_rcv_space_adjust+0xbe/0x160
[63567.802449] ? tcp_rcv_space_adjust+0x1a/0x160
[63567.802456] tcp_recvmsg_locked+0x2d4/0x960
[63567.802464] tcp_recvmsg+0x87/0x1f0
[63567.802471] inet6_recvmsg+0x56/0x130
[63567.802479] ? __pfx_bpf_lsm_socket_recvmsg+0x10/0x10
[63567.802488] ? security_socket_recvmsg+0x44/0x70
[63567.802497] sock_recvmsg+0x57/0xd0
[63567.802505] sock_read_iter+0x96/0x100
[63567.802513] vfs_read+0x303/0x350
[63567.802796] ksys_read+0xbb/0xf0
[63567.803074] do_syscall_64+0x60/0x90
[63567.803346] ? syscall_exit_to_user_mode+0x2b/0x40
[63567.803619] ? do_syscall_64+0x6c/0x90
[63567.803890] ? syscall_exit_to_user_mode+0x2b/0x40
[63567.804163] ? do_syscall_64+0x6c/0x90
[63567.804438] ? do_syscall_64+0x6c/0x90
[63567.804710] entry_SYSCALL_64_after_hwframe+0x6e/0xd8
[63567.804984] RIP: 0033:0x7fc3d5f21531
[63567.805271] Code: ff ff eb c3 67 e8 2f c9 01 00 66 2e 0f 1f 84 00 00
00 00 00 0f 1f 44 00 00 f3 0f 1e fa 80 3d 35 ce 0d 00 00 74 13 31 c0 0f
05 <48> 3d 00 f0 ff ff
77 57 c3 66 0f 1f 44 00 00 48 83 ec 28 48 89 54
[63567.805909] RSP: 002b:00007fffa66f6748 EFLAGS: 00000246 ORIG_RAX:
0000000000000000
[63567.806225] RAX: ffffffffffffffda RBX: 00007fffa66f6830 RCX:
00007fc3d5f21531
[63567.806542] RDX: 000000000000191a RSI: 000055d106003d80 RDI:
00000000000000bd
[63567.806855] RBP: 000055d10620bc30 R08: 000055d10620bc30 R09:
000055d1036a8e78
[63567.807163] R10: 000055d1037ad630 R11: 0000000000000246 R12:
000000000000191a
[63567.807472] R13: 000055d10664f410 R14: 000055d106003d80 R15:
00007fc3d627c6b8
[63567.807768] </TASK>
[63567.808037] Modules linked in: nfnetlink_queue nf_conntrack_netlink
xt_connmark xt_mark ip_set_hash_net xt_NFQUEUE xt_set ip_set_hash_ip
ip_set cls_fw sch_sfq sch_h
tb nf_nat_ftp nf_conntrack_ftp tls tun cfg80211 rfkill nft_chain_nat
xt_MASQUERADE xt_nat xt_REDIRECT nf_nat nft_limit xt_limit xt_conntrack
nf_conntrack nf_defrag_ipv
6 nf_defrag_ipv4 xt_multiport xt_tcpudp nft_compat nf_tables libcrc32c
intel_rapl_msr intel_rapl_common x86_pkg_temp_thermal intel_powerclamp
snd_hda_codec_realtek cor
etemp snd_hda_codec_generic ledtrig_audio snd_hda_codec_hdmi kvm_intel
snd_hda_intel kvm snd_intel_dspcfg snd_intel_sdw_acpi irqbypass
snd_hda_codec snd_hda_core crct1
0dif_pclmul r8169 crc32_pclmul snd_hwdep polyval_clmulni snd_pcm
polyval_generic gf128mul snd_timer realtek ax88179_178a spi_nor
mdio_devres usbnet snd libphy ghash_cl
mulni_intel mtd sha512_ssse3 at24 soundcore mii mei_pxp mei_hdcp
sha256_ssse3 sha1_ssse3 iTCO_wdt i2c_i801 mei_me spi_intel_platform
mousedev aesni_intel intel_pmc_bxt
spi_intel iTCO_vendor_support
[63567.808076] i2c_smbus mei crypto_simd lpc_ich cryptd rapl
intel_cstate intel_uncore psmouse mac_hid pcspkr pkcs8_key_parser fuse
dm_mod loop nfnetlink ip_tables x_
tables ext4 crc32c_generic crc16 mbcache jbd2 i915 serio_raw
i2c_algo_bit atkbd drm_buddy libps2 ttm vivaldi_fmap intel_gtt xhci_pci
crc32c_intel drm_display_helper xh
ci_pci_renesas cec i8042 video serio wmi
[63567.811333] ---[ end trace 0000000000000000 ]---
[63567.811701] RIP: 0010:tcp_rcv_space_adjust+0xbe/0x160
[63567.812069] Code: f8 41 89 d0 29 d0 31 d2 49 0f af c2 49 f7 f0 45 8b
81 bc 04 00 00 44 0f b6 8b 10 06 00 00 31 d2 49 8d 04 42 48 98 48 c1 e0
08 <49> f7 f1 49 63 d0
48 98 48 39 d0 48 0f 47 c2 39 83 18 01 00 00 7c
[63567.812848] RSP: 0018:ffffba9f00da3c18 EFLAGS: 00010206
[63567.813252] RAX: 0000000004fb2800 RBX: ffff9e124acc1c80 RCX:
0000000eccd9dace
[63567.813651] RDX: 0000000000000000 RSI: 0000000037fa4ae8 RDI:
0000000000008349
[63567.814064] RBP: ffff9e124acc1c80 R08: 0000000000600000 R09:
0000000000000000
[63567.814470] R10: 00000000000161d2 R11: ffff9e1346621700 R12:
0000000000000000
[63567.814879] R13: ffff9e124acc1d58 R14: 0000000000000000 R15:
ffff9e124acc2214
[63567.815290] FS: 00007fc3d627c840(0000) GS:ffff9e1457380000(0000)
knlGS:0000000000000000
[63567.815700] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[63567.816121] CR2: 0000560b7e86d690 CR3: 0000000102f44001 CR4:
00000000001706e0
After this the "child" squid process can not be killed, even by systemd,
even by kill -9.
It appears that squid "main" process (owned by root user) does not hang
and hence connections on port 8080 are accepted.
But squid child process (owned by proxy user) hangs and never recovers
after above dmesg. And hence nothing happens and connection times out.
# ps aux |grep squid
root 612 0.0 0.2 75500 23680 ? Ss Dec20 0:02
/usr/bin/squid -f /etc/squid/btnet/squid.btnet.conf --foreground -sYC
proxy 617 0.0 0.0 0 0 ? D Dec20 1:00 [squid]
proxy 620 0.0 0.0 12152 7552 ? S Dec20 0:00
(security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB
proxy 621 0.0 0.0 12152 7552 ? S Dec20 0:00
(security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB
proxy 622 0.0 0.0 11888 5504 ? S Dec20 0:00
(security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB
proxy 623 0.0 0.0 11888 5632 ? S Dec20 0:00
(security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB
proxy 624 0.0 0.0 11888 5632 ? S Dec20 0:00
(security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB
proxy 643 0.0 0.0 6116 3200 ? S Dec20 0:01
(logfile-daemon) /var/log/squid/access.log
# telnet 127.0.0.1 8080
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
^]
telnet> c
# curl -x 127.0.0.1:8080 --max-time 30 https://google.com
curl: (28) Connection timed out after 30002 milliseconds
# kill 612 617
Ran kill 5 times, nothing happened. All squid processes remained.
# kill -9 612 617
Ran once, and all squid processes except 617 (child squid process) got
killed
# ps aux |grep squid
proxy 617 0.0 0.0 0 0 ? D Dec20 1:00 [squid]
root 90764 0.0 0.0 6552 2560 pts/5 S+ 17:36 0:00 grep
--color=auto squid
Can above information be of any help?
Thanks and regards,
Amish
On 19/12/23 20:46, Alex Rousskov wrote:
> On 2023-12-18 22:29, Amish wrote:
>> On 19/12/23 01:14, Alex Rousskov wrote:
>>> On 2023-12-18 09:35, Amish wrote:
>>>
>>>> I use Arch Linux and today I updated squid from squid 5.7 to squid
>>>> 6.6.
>>>
>>> > Dec 18 13:01:24 mumbai squid[604]: kick abandoning conn199
>>>
>>> I do not know whether the above problem is the primary problem in
>>> your setup, but it is a red flag. Transactions on the same
>>> connection may get stuck after that message; it is essentially a
>>> Squid bug.
>>>
>>> I am not sure at all, but this bug might be related to Bug 5187
>>> workaround that went into Squid v6.2 (commit c44cfe7):
>>> https://bugs.squid-cache.org/show_bug.cgi?id=5187
>>>
>>> Does Squid accept new TCP connections after it enters what you
>>> describe as a dead state? For example, does "telnet 127.0.0.1 8080"
>>> establishes a connection if executed on the same machine as Squid?
>>>
>> Yes it establishes connection. But I do not know what to do next.
>
> This tells us that your Squid is still listening for incoming
> connections. Most likely, it is not "dead" but running and just unable
> to make progress with those connections (for yet unknown reasons).
> That information is helpful but not sufficient (for me) to solve the
> problem you are describing.
>
> The next step that I would recommend is to collect debugging
> information from the running process and share a pointer to the
> corresponding compressed cache.log file:
>
> * Ideally, start collection when Squid starts and reproduce the
> problem while collecting full debugging information:
> http://wiki.squid-cache.org/SquidFaq/BugReporting#full-debug-output
>
> * If you have to, start collection after Squid is already in bad state
> and just before you use telnet or browser to tickle Squid:
> http://wiki.squid-cache.org/SquidFaq/BugReporting#debugging-a-single-transaction
>
>
> Do not use any secret information (e.g., production certificate keys)
> for these tests (unless you are going to share the logs privately with
> those you trust).
>
> Do not downgrade to v5 for these tests.
>
>
> HTH,
>
> Alex.
>
>
>> Browser showed "Connection timed out" message. But I believe
>> browser's also connected but nothing happened afterwards.
>>
>>>
>>> > kill -9 does nothing
>>>
>>> Is it possible that you are trying to kill the wrong process? You
>>> should be killing this process AFAICT:
>>>
>>> > root 601 0.0 0.2 73816 22528 ? Ss 12:59 0:02
>>> > /usr/bin/squid -f /etc/squid/btnet/squid.btnet.conf --foreground -sYC
>>
>> I did not clarify but all processes needed SIGKILL and vanished
>> except the Dead squid process which still remained.
>>
>> # systemctl stop squid
>>
>> Dec 19 08:46:38 mumbai systemd[1]: squid.service: State
>> 'stop-sigterm' timed out. Killing.
>> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 601
>> (squid) with signal SIGKILL.
>> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 604
>> (squid) with signal SIGKILL.
>> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 607
>> (security_file_c) with signal SIGKILL.
>> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 608
>> (security_file_c) with signal SIGKILL.
>> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 609
>> (security_file_c) with signal SIGKILL.
>> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 610
>> (security_file_c) with signal SIGKILL.
>> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 611
>> (security_file_c) with signal SIGKILL.
>> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 622
>> (log_file_daemon) with signal SIGKILL.
>> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Main process
>> exited, code=killed, status=9/KILL
>> Dec 19 08:46:38 mumbai systemd[1]: squid.service: Killing process 604
>> (squid) with signal SIGKILL.
>>
>> Waited for 2 minutes for squid to stop then pressed ctrl-c to
>> systemctl stop squid command.
>>
>> As you can see in last line shows that attempt was made to kill DEAD
>> process with PID 604.
>>
>> # ps aux |grep squid
>> proxy 604 0.0 0.0 0 0 ? D Dec18 0:03 [squid]
>>
>> Now only DEAD squid process remains.
>>
>> What next? Should I downgrade to 5.9 and check?
>>
>> Regards
>>
>> Amish
>>
>>> Alex.
>>>
>>>
>>>> After the update from 5.7 to 6.6, squid starts but then reaches
>>>> Dead state in a minute or two.
>>>>
>>>> # ps aux | grep squid
>>>> root 601 0.0 0.2 73816 22528 ? Ss 12:59 0:02
>>>> /usr/bin/squid -f /etc/squid/btnet/squid.btnet.conf --foreground -sYC
>>>> proxy 604 0.0 0.0 0 0 ? D 12:59 0:03
>>>> [squid]
>>>> proxy 607 0.0 0.0 11976 7424 ? S 12:59 0:00
>>>> (security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB
>>>> proxy 608 0.0 0.0 11976 7168 ? S 12:59 0:00
>>>> (security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB
>>>> proxy 609 0.0 0.0 11712 5632 ? S 12:59 0:00
>>>> (security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB
>>>> proxy 610 0.0 0.0 11712 5376 ? S 12:59 0:00
>>>> (security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB
>>>> proxy 611 0.0 0.0 11712 5504 ? S 12:59 0:00
>>>> (security_file_certgen) -s /var/cache/squid/ssl_db -M 4MB
>>>> proxy 622 0.0 0.0 6116 3200 ? S 12:59 0:00
>>>> (logfile-daemon) /var/log/squid/access.log
>>>>
>>>> And then all requests get stuck. Notice the D (dead) state of squid.
>>>>
>>>> I use multiple ports for multiple purposes. (It all worked fine in
>>>> squid 5.7)
>>>>
>>>> Dec 18 12:59:10 mumbai squid[601]: Starting Authentication on port
>>>> [::]:3128
>>>> Dec 18 12:59:10 mumbai squid[601]: Disabling Authentication on port
>>>> [::]:3128 (interception enabled)
>>>> Dec 18 12:59:10 mumbai squid[601]: Starting Authentication on port
>>>> [::]:8081
>>>> Dec 18 12:59:10 mumbai squid[601]: Disabling Authentication on port
>>>> [::]:8081 (interception enabled)
>>>> Dec 18 12:59:12 mumbai squid[601]: Starting Authentication on port
>>>> [::]:8082
>>>> Dec 18 12:59:12 mumbai squid[601]: Disabling Authentication on port
>>>> [::]:8082 (interception enabled)
>>>> Dec 18 12:59:12 mumbai squid[601]: Starting Authentication on port
>>>> [::]:8083
>>>> Dec 18 12:59:12 mumbai squid[601]: Disabling Authentication on port
>>>> [::]:8083 (interception enabled)
>>>> Dec 18 12:59:13 mumbai squid[601]: Starting Authentication on port
>>>> [::]:8084
>>>> Dec 18 12:59:13 mumbai squid[601]: Disabling Authentication on port
>>>> [::]:8084 (interception enabled)
>>>> Dec 18 12:59:13 mumbai squid[601]: Starting Authentication on port
>>>> [::]:3136
>>>> Dec 18 12:59:13 mumbai squid[601]: Disabling Authentication on port
>>>> [::]:3136 (interception enabled)
>>>> Dec 18 12:59:13 mumbai squid[601]: Starting Authentication on port
>>>> [::]:3137
>>>> Dec 18 12:59:13 mumbai squid[601]: Disabling Authentication on port
>>>> [::]:3137 (interception enabled)
>>>> ...
>>>> Dec 18 12:59:29 mumbai squid[604]: Adaptation support is on
>>>> Dec 18 12:59:29 mumbai squid[604]: Accepting NAT intercepted HTTP
>>>> Socket connections at conn19 local=[::]:3128 remote=[::] FD 27
>>>> flags=41
>>>> listening port: 3128
>>>> Dec 18 12:59:29 mumbai squid[604]: Accepting SSL bumped HTTP Socket
>>>> connections at conn21 local=[::]:8080 remote=[::] FD 28 flags=9
>>>> listening port: 8080
>>>> Dec 18 12:59:29 mumbai squid[604]: Accepting NAT intercepted SSL
>>>> bumped HTTPS Socket connections at conn23 local=[::]:8081
>>>> remote=[::] FD 29 flags=41
>>>> listening port: 8081
>>>> Dec 18 12:59:29 mumbai squid[604]: Accepting SSL bumped HTTP Socket
>>>> connections at conn25 local=[::]:8092 remote=[::] FD 30 flags=9
>>>> listening port: 8092
>>>> Dec 18 12:59:29 mumbai systemd[1]: Started Squid Web Proxy Server.
>>>> Dec 18 12:59:29 mumbai squid[604]: Accepting SSL bumped HTTP Socket
>>>> connections at conn27 local=[::]:8093 remote=[::] FD 31 flags=9
>>>> listening port: 8093
>>>> Dec 18 12:59:29 mumbai squid[604]: Accepting SSL bumped HTTP Socket
>>>> connections at conn29 local=[::]:8094 remote=[::] FD 32 flags=9
>>>> listening port: 8094
>>>> Dec 18 12:59:29 mumbai squid[604]: Accepting NAT intercepted SSL
>>>> bumped HTTPS Socket connections at conn31 local=[::]:8082
>>>> remote=[::] FD 33 flags=41
>>>> listening port: 8082
>>>> Dec 18 12:59:29 mumbai squid[604]: Accepting NAT intercepted SSL
>>>> bumped HTTPS Socket connections at conn33 local=[::]:8083
>>>> remote=[::] FD 34 flags=41
>>>> listening port: 8083
>>>> Dec 18 12:59:29 mumbai squid[604]: Accepting NAT intercepted SSL
>>>> bumped HTTPS Socket connections at conn35 local=[::]:8084
>>>> remote=[::] FD 35 flags=41
>>>> listening port: 8084
>>>> Dec 18 12:59:29 mumbai squid[604]: Accepting NAT intercepted HTTP
>>>> Socket connections at conn37 local=[::]:3136 remote=[::] FD 36
>>>> flags=41
>>>> listening port: 3136
>>>> Dec 18 12:59:29 mumbai squid[604]: Accepting NAT intercepted HTTP
>>>> Socket connections at conn39 local=[::]:3137 remote=[::] FD 37
>>>> flags=41
>>>> listening port: 3137
>>>>
>>>> And then following errors came:
>>>>
>>>>
>>>> Dec 18 12:59:45 mumbai squid[604]: ERROR: failure while accepting a
>>>> TLS connection on conn41 local=192.168.0.1:8080
>>>> remote=192.168.0.111:53867 FD 12 flags=1: SQUID_TLS
>>>> _ERR_ACCEPT+TLS_LIB_ERR=A000416+TLS_IO_ERR=1
>>>> current master transaction:
>>>> master53
>>>> Dec 18 12:59:45 mumbai squid[604]: ERROR: failure while accepting a
>>>> TLS connection on conn42 local=192.168.0.1:8080
>>>> remote=192.168.0.111:53868 FD 14 flags=1: SQUID_TLS
>>>> _ERR_ACCEPT+TLS_LIB_ERR=A000416+TLS_IO_ERR=1
>>>> current master transaction:
>>>> master53
>>>> Dec 18 12:59:45 mumbai squid[604]: ERROR: failure while accepting a
>>>> TLS connection on conn43 local=192.168.0.1:8080
>>>> remote=192.168.0.111:53869 FD 16 flags=1: SQUID_TLS
>>>> _ERR_ACCEPT+TLS_LIB_ERR=A000416+TLS_IO_ERR=1
>>>> current master transaction:
>>>> master57
>>>> Dec 18 12:59:45 mumbai squid[604]: ERROR: failure while accepting a
>>>> TLS connection on conn44 local=192.168.0.1:8080
>>>> remote=192.168.0.111:53870 FD 12 flags=1: SQUID_TLS
>>>> _ERR_ACCEPT+TLS_LIB_ERR=A000416+TLS_IO_ERR=1
>>>> current master transaction:
>>>> master57
>>>> Dec 18 12:59:56 mumbai squid[604]: ERROR: failure while accepting a
>>>> TLS connection on conn62 local=192.168.0.1:8080
>>>> remote=192.168.0.111:53887 FD 12 flags=1: SQUID_TLS
>>>> _ERR_ACCEPT+TLS_LIB_ERR=A000416+TLS_IO_ERR=1
>>>> current master transaction:
>>>> master95
>>>> Dec 18 12:59:59 mumbai squid[604]: ERROR: failure while accepting a
>>>> TLS connection on conn64 local=192.168.0.1:8080
>>>> remote=192.168.0.111:53888 FD 12 flags=1: SQUID_TLS
>>>> _ERR_ACCEPT+TLS_LIB_ERR=A000416+TLS_IO_ERR=1
>>>> current master transaction:
>>>> master99
>>>> Dec 18 13:00:02 mumbai squid[604]: ERROR: failure while accepting a
>>>> TLS connection on conn65 local=192.168.0.1:8080
>>>> remote=192.168.0.178:56115 FD 12 flags=1: SQUID_TLS
>>>> _ERR_ACCEPT+TLS_LIB_ERR=A000418+TLS_IO_ERR=1
>>>> current master transaction:
>>>> master53
>>>> Dec 18 13:01:24 mumbai squid[604]: kick abandoning conn199
>>>> local=192.168.0.1:8093 remote=192.168.0.101:52211 FD 52 flags=1
>>>> connection: conn199
>>>> local=192.168.0.1:8093 remote=192.168.0.101:52211 FD 52 flags=1
>>>> Dec 18 13:01:45 mumbai squid[604]: ERROR: failure while accepting a
>>>> TLS connection on conn240 local=192.168.0.1:8093
>>>> remote=192.168.0.111:53931 FD 48 flags=1: SQUID_TL
>>>> S_ERR_ACCEPT+TLS_LIB_ERR=A000416+TLS_IO_ERR=1
>>>> current master transaction:
>>>> master314
>>>>
>>>>
>>>> After this point there is nothing in systemd journal (via:
>>>> journalctl -f -u squid) and same lines are in cache.log.
>>>>
>>>> Squid got stuck (DEAD state) at 13:01 and right now it 19:26 (6
>>>> hours passed) and squid is still in dead state.
>>>>
>>>> kill -9 or kill -ALRM or -HUP also does nothing.
>>>>
>>>> So to restart squid - I will need to restart whole system.
>>>>
>>>> I have sslbump directives but it is not really applied.
>>>>
>>>> #NOTE: nosslbump_ips below contains 192.168.0.0/24 (whole LAN) so
>>>> effectively there is no SSL bump after step1.
>>>>
>>>> acl nosslbump_ips src 192.168.0.0/24
>>>> ssl_bump splice ssl_step1 nosslbump_ips
>>>> ssl_bump peek ssl_step1
>>>> ssl_bump splice nosslbump_domains
>>>> ssl_bump stare sslbump_domains
>>>> ssl_bump splice ssl_step2
>>>> ssl_bump bump all
>>>>
>>>>
>>>> Any idea? If anything changed from 5.7 to 6.6 that may cause this
>>>> behaviour?
>>>>
>>>> Looking at changelog:
>>>>
>>>> Bug 5256: Intercepting port fails to accept
>>>> https://bugs.squid-cache.org/show_bug.cgi?id=5256
>>>>
>>>> Bug 5154: Do not open IPv6 sockets when IPv6 is disabled
>>>> https://bugs.squid-cache.org/show_bug.cgi?id=5154
>>>>
>>>> Not sure if above two bug FIXES (in between v5.7 to v6.6) are
>>>> related to my issue.
>>>>
>>>> I ran netstat:
>>>>
>>>> # netstat -ntlp
>>>> Active Internet connections (only servers)
>>>> Proto Recv-Q Send-Q Local Address Foreign Address
>>>> State PID/Program name
>>>> ...
>>>> tcp6 33 0 :::3137 :::* LISTEN -
>>>> tcp6 0 0 :::3136 :::* LISTEN -
>>>> tcp6 4 0 :::3128 :::* LISTEN -
>>>> tcp6 0 0 :::8081 :::* LISTEN -
>>>> tcp6 0 0 :::8080 :::* LISTEN -
>>>> tcp6 0 0 :::8083 :::* LISTEN -
>>>> tcp6 0 0 :::8082 :::* LISTEN -
>>>> tcp6 0 0 :::8084 :::* LISTEN -
>>>> tcp6 4097 0 :::8093 :::* LISTEN -
>>>> tcp6 0 0 :::8092 :::* LISTEN -
>>>> tcp6 0 0 :::8094 :::* LISTEN -
>>>> ...
>>>>
>>>> I do not have IPv6 enabled, yet there are 33 and 4097 numbers in
>>>> Recv-Q and also no process/PID owns these ports.
>>>>
>>>> Same IPv4 ports are not shown in use by netstat, so only IPv6 ports
>>>> remain open, that too orphaned!
>>>>
>>>> So what is happening?
>>>>
>>>> Any idea to solve or any workaround?
>>>>
>>>> Thank you,
>>>>
>>>> Amish.
>> _______________________________________________
>> squid-users mailing list
>> squid-users at lists.squid-cache.org
>> https://lists.squid-cache.org/listinfo/squid-users
>
More information about the squid-users
mailing list