[squid-users] All Adaptation ICAPs go down at the same time

[Alex Rousskov]

On 5/23/21 2:05 AM, roie rachamim wrote:

> Patch seems to do the trick,
> When is it expected to be merged ?

It will be merged into master/v6 in a few hours AFAICT. You can track
status using the PR 795 link.

Alex.


> On Thu, May 20, 2021 at 12:53 AM Alex Rousskov wrote:
> 
>     On 5/19/21 5:31 PM, roie rachamim wrote:
> 
>     > 2021/05/12 12:27:24.209| 93,5| AsyncJob.cc(139) callEnd:
>     > AsyncJob::start() ends job [/ job31640]
> 
>     To me, this looks like bug 4528:
>     https://bugs.squid-cache.org/show_bug.cgi?id=4528
>     <https://bugs.squid-cache.org/show_bug.cgi?id=4528>
> 
>     That bug is being fixed in PR 795:
>     https://github.com/squid-cache/squid/pull/795
>     <https://github.com/squid-cache/squid/pull/795>
> 
>     You might be able to patch your Squid using the corresponding patch:
>     https://github.com/squid-cache/squid/pull/795.diff
>     <https://github.com/squid-cache/squid/pull/795.diff>
> 
>     Testing feedback is very weclomed.
> 
> 
>     HTH,
> 
>     Alex.
> 
> 
>     > On Mon, Apr 19, 2021 at 12:07 PM Eliezer Croitoru wrote:
>     >
>     >     Hey Roie,____
>     >
>     >     __ __
>     >
>     >     From the output I assume it’s a dns resolution issue.____
>     >
>     >     In the past I remember that Docker was updating the hosts file
>     with
>     >     the relevant names but  it’s not working the same way now.____
>     >
>     >     Currently Docker is using a local network dns service which is
>     being
>     >     accessed via 127.0.0.53.____
>     >
>     >     From I remember Squid is resolving the icap service name only at
>     >     startup or reload.____
>     >
>     >     Lately Alex published a testable patch that might fix specific
>     >     issues with icap services which are resolved by dns. ( sorry I
>     don’t
>     >     remember the bug report)____
>     >
>     >     I assume you can try to test this patch first.____
>     >
>     >     If these services are static to some degree you might be able to
>     >     create a script that updates the hosts file and reload squid
>     on each
>     >     change.____
>     >
>     >     When using the hosts file it’s possible that some issues will
>     >     disappear.____
>     >
>     >
>     >     There is also another possibility which is a malformed ICAP
>     response
>     >     or wrong sessions handling which cause this issue.____
>     >
>     >     You might be able to use tcpdump from either the host or the
>     >     container side to capture traffic when these goes down.____
>     >
>     >     Depends on your preference of debug level you might even be
>     able to
>     >     debug specific debug_options like for ICAP services
>     >     and/or requests to the degree you might be able to see what
>     happens
>     >     on the basic level of the ICAP encapsulation.____
>     >
>     >     If you really need help with a diagnosis and a solution you
>     might be
>     >     able to use Alex and the measurement factory.
>     >
>     >     ____
>     >
>     >     All The Bests,____
>     >
>     >     Eliezer____
>     >
>     >     __ __
>     >
>     >     *From:* squid-users <squid-users-bounces at lists.squid-cache.org
>     <mailto:squid-users-bounces at lists.squid-cache.org>
>     >     <mailto:squid-users-bounces at lists.squid-cache.org
>     <mailto:squid-users-bounces at lists.squid-cache.org>>> *On Behalf Of
>     >     *roie rachamim
>     >     *Sent:* Monday, April 12, 2021 12:54 PM
>     >     *To:* squid-users at lists.squid-cache.org
>     <mailto:squid-users at lists.squid-cache.org>
>     >     <mailto:squid-users at lists.squid-cache.org
>     <mailto:squid-users at lists.squid-cache.org>>
>     >     *Subject:* [squid-users] All Adaptation ICAPs go down at the same
>     >     time____
>     >
>     >     __ __
>     >
>     >     Hi,____
>     >
>     >     __ __
>     >
>     >     Our setup includes squid that runs in docker container with
>     several
>     >     ICAP servers in additional containers.____
>     >
>     >     From time to time we see in cache.log the following messages:
>     >     2021/04/12 00:22:39| optional ICAP service is down after an
>     options
>     >     fetch failure: icap://icap1.proxy:14590/censor [down,!opt]
>     >     2021/04/12 00:22:39| optional ICAP service is down after an
>     options
>     >     fetch failure: icap://icap2.proxy:1344/request [down,!opt]
>     >     2021/04/12 00:22:39| optional ICAP service is down after an
>     options
>     >     fetch failure: icap://icap3.proxy:14590/response [down,!opt]
>     >
>     >     2021/04/12 06:10:45| optional ICAP service is down after an
>     options
>     >     fetch failure: icap://icap1.proxy:14590/censor [down,!opt]
>     >     2021/04/12 06:10:45| optional ICAP service is down after an
>     options
>     >     fetch failure: icap://icap2.proxy:1344/request [down,!opt]
>     >     2021/04/12 06:10:45| optional ICAP service is down after an
>     options
>     >     fetch failure: icap://icap3.proxy:14590/response [down,!opt]____
>     >
>     >     __ __
>     >
>     >     We're trying to understand why it happens to all ICAPs at
>     once. This
>     >     happens in 4.14 and in 5.0.4____
>     >
>     >     Any thoughts about what might cause this ?____
>     >
>     >     Many Thanks,____
>     >
>     >     Roie____
>     >
>     >
>     > _______________________________________________
>     > squid-users mailing list
>     > squid-users at lists.squid-cache.org
>     <mailto:squid-users at lists.squid-cache.org>
>     > http://lists.squid-cache.org/listinfo/squid-users
>     <http://lists.squid-cache.org/listinfo/squid-users>
>     >
>