[squid-dev] Rock store stopped accessing discs

[Alex Rousskov]

On 03/17/2017 09:48 AM, Heiler Bemerguy wrote:
> Sadly the same thing occurred again. It seems the hole is deeper lol..

Most likely, it is the same hole. However, the more we panic and jump to
conclusions, the deeper that hole below us may look...

> I couldn't find any previous messages that could give a clue why this is happening..

> 2017/03/15 19:01:29 kid2| WARNING: communication with /cache2/rock may be too slow or disrupted for about 7.00s; rescued 1 out of 1 I/Os
> 2017/03/15 19:15:05 kid2| WARNING: communication with /cache2/rock may be too slow or disrupted for about 7.00s; rescued 1 out of 1 I/Os
> 2017/03/15 19:18:02 kid2| WARNING: communication with /cache2/rock may be too slow or disrupted for about 7.00s; rescued 1 out of 1 I/Os
> 2017/03/15 19:22:27 kid2| WARNING: communication with /cache2/rock may be too slow or disrupted for about 7.00s; rescued 1 out of 1 I/Os
> 2017/03/15 19:26:19 kid2| WARNING: communication with /cache2/rock may be too slow or disrupted for about 7.00s; rescued 1 out of 1 I/Os

In general, such semi-periodic problems without any other serious error
messages could be due to missing max-swap-rate which is designed to
prevent disk overload and associated process blockage by the OS. I doubt
that is what happening in your particular case because I would expect
more I/Os to be affected (but you can validate that theory by following
my original recommendation).

Can you reproduce this problem if you increase logging levels? If yes, I
suggest configuring each kid to log to its own cache-N.log file and then
use "-k debug" (or equivalent) to collect detailed logs containing a few
of these WARNINGs.


Cheers,

Alex.