[squid-users] Too many ERROR: Collapsed forwarding queue overflow for kid2 at 1024 items

[Loučanský Lukáš]

I'm running Squid Object Cache: Version 6.0.0-20211116-r90086c5a8 for about 14hrs now. I've noticed many new  "ERROR: Collapsed forwarding queue overflow for kid2 at 1024 items" after about 11hrs runtime.
And I have
Transients queues:
  kid1 receiving from kid1: { size: 268, capacity: 1024, other: 1198, popIndex: 930, items: [
    { sender: 1, xitIndex: 5801 },
    { sender: 1, xitIndex: 1322 },
    { sender: 1, xitIndex: 6790 },
    # ... 262 items not shown ...
    { sender: 1, xitIndex: 1928 },
    { sender: 1, xitIndex: 7030 },
    { sender: 1, xitIndex: 6297 },
  ]}
  kid1 receiving from kid2: { size: 1024, capacity: 1024, other: 60523, popIndex: 59499, items: [
    { sender: 2, xitIndex: 4481 },
    { sender: 2, xitIndex: 4481 },
    { sender: 2, xitIndex: 3277 },
    # ... 1018 items not shown ...
    { sender: 2, xitIndex: 12520 },
    { sender: 2, xitIndex: 12520 },
    { sender: 2, xitIndex: 12520 },
  ]}

  kid1 sending to kid1: { size: 268, capacity: 1024, pushIndex: 1198, other: 930, items: [
    { sender: 1, xitIndex: 5801 },
    { sender: 1, xitIndex: 1322 },
    { sender: 1, xitIndex: 6790 },
    # ... 262 items not shown ...
    { sender: 1, xitIndex: 1928 },
    { sender: 1, xitIndex: 7030 },
    { sender: 1, xitIndex: 6297 },
  ]}
  kid1 sending to kid2: { size: 1024, capacity: 1024, pushIndex: 39566, other: 38542, items: [
    { sender: 1, xitIndex: 13750 },
    { sender: 1, xitIndex: 608 },
    { sender: 1, xitIndex: 608 },
    # ... 1018 items not shown ...
    { sender: 1, xitIndex: 14740 },
    { sender: 1, xitIndex: 14740 },
    { sender: 1, xitIndex: 14740 },
  ]}

Similar for kid2.

cat cache.log | grep assert
2021/11/22 07:54:24 kid2| assertion failed: store.cc:1094: "store_status == STORE_PENDING"
2021/11/22 08:51:48 kid1| assertion failed: store.cc:1094: "store_status == STORE_PENDING"

(maybe consequence of too generous max-swap-rate)
LL

-----Original Message-----
From: Alex Rousskov [mailto:rousskov at measurement-factory.com] 
Sent: Sunday, November 21, 2021 8:28 PM
To: Loučanský Lukáš; squid-users at lists.squid-cache.org
Subject: Re: [squid-users] Too many ERROR: Collapsed forwarding queue overflow for kid2 at 1024 items

On 11/21/21 2:08 PM, Loučanský Lukáš wrote:
> Hello, I've replaced src/ipc/ReadWritelock.cc and ReadWriteLock.h with 
> modified versions (with finalizeExclusive calls), but afeter some time 
> I have got queue overflows agains.

After each bug fix, we have to restart the triage sequence from scratch:
Any assertions, crashes, FATAL messages or similar things leading to kid deaths? If they are present, then they explain queue overflows.


> So I've downloaded squid-6.0.0-20211116-r90086c5a8 - run it with my v5 
> configure and now I'm testing it. So far I get these:
> 2021/11/21 19:05:06 kid1| BUG: missing ENTRY_REQUIRES_COLLAPSING for

Noted. If you cannot reproduce this bug at will, then I recommend focusing on other, easier-to-address problems first.

Alex.