[squid-users] All Adaptation ICAPs go down at the same time

[Alex Rousskov]

On 5/19/21 5:31 PM, roie rachamim wrote:

> 2021/05/12 12:27:24.209| 93,5| AsyncJob.cc(139) callEnd:
> AsyncJob::start() ends job [/ job31640]

To me, this looks like bug 4528:
https://bugs.squid-cache.org/show_bug.cgi?id=4528

That bug is being fixed in PR 795:
https://github.com/squid-cache/squid/pull/795

You might be able to patch your Squid using the corresponding patch:
https://github.com/squid-cache/squid/pull/795.diff

Testing feedback is very weclomed.


HTH,

Alex.


> On Mon, Apr 19, 2021 at 12:07 PM Eliezer Croitoru wrote:
> 
>     Hey Roie,____
> 
>     __ __
> 
>     From the output I assume it’s a dns resolution issue.____
> 
>     In the past I remember that Docker was updating the hosts file with
>     the relevant names but  it’s not working the same way now.____
> 
>     Currently Docker is using a local network dns service which is being
>     accessed via 127.0.0.53.____
> 
>     From I remember Squid is resolving the icap service name only at
>     startup or reload.____
> 
>     Lately Alex published a testable patch that might fix specific
>     issues with icap services which are resolved by dns. ( sorry I don’t
>     remember the bug report)____
> 
>     I assume you can try to test this patch first.____
> 
>     If these services are static to some degree you might be able to
>     create a script that updates the hosts file and reload squid on each
>     change.____
> 
>     When using the hosts file it’s possible that some issues will
>     disappear.____
> 
> 
>     There is also another possibility which is a malformed ICAP response
>     or wrong sessions handling which cause this issue.____
> 
>     You might be able to use tcpdump from either the host or the
>     container side to capture traffic when these goes down.____
> 
>     Depends on your preference of debug level you might even be able to
>     debug specific debug_options like for ICAP services
>     and/or requests to the degree you might be able to see what happens
>     on the basic level of the ICAP encapsulation.____
> 
>     If you really need help with a diagnosis and a solution you might be
>     able to use Alex and the measurement factory.
> 
>     ____
> 
>     All The Bests,____
> 
>     Eliezer____
> 
>     __ __
> 
>     *From:* squid-users <squid-users-bounces at lists.squid-cache.org
>     <mailto:squid-users-bounces at lists.squid-cache.org>> *On Behalf Of
>     *roie rachamim
>     *Sent:* Monday, April 12, 2021 12:54 PM
>     *To:* squid-users at lists.squid-cache.org
>     <mailto:squid-users at lists.squid-cache.org>
>     *Subject:* [squid-users] All Adaptation ICAPs go down at the same
>     time____
> 
>     __ __
> 
>     Hi,____
> 
>     __ __
> 
>     Our setup includes squid that runs in docker container with several
>     ICAP servers in additional containers.____
> 
>     From time to time we see in cache.log the following messages:
>     2021/04/12 00:22:39| optional ICAP service is down after an options
>     fetch failure: icap://icap1.proxy:14590/censor [down,!opt]
>     2021/04/12 00:22:39| optional ICAP service is down after an options
>     fetch failure: icap://icap2.proxy:1344/request [down,!opt]
>     2021/04/12 00:22:39| optional ICAP service is down after an options
>     fetch failure: icap://icap3.proxy:14590/response [down,!opt]
> 
>     2021/04/12 06:10:45| optional ICAP service is down after an options
>     fetch failure: icap://icap1.proxy:14590/censor [down,!opt]
>     2021/04/12 06:10:45| optional ICAP service is down after an options
>     fetch failure: icap://icap2.proxy:1344/request [down,!opt]
>     2021/04/12 06:10:45| optional ICAP service is down after an options
>     fetch failure: icap://icap3.proxy:14590/response [down,!opt]____
> 
>     __ __
> 
>     We're trying to understand why it happens to all ICAPs at once. This
>     happens in 4.14 and in 5.0.4____
> 
>     Any thoughts about what might cause this ?____
> 
>     Many Thanks,____
> 
>     Roie____
> 
> 
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>