<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple style='word-wrap:break-word'><div class=WordSection1><p class=MsoNormal>Hey Roie,<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>From the output I assume it’s a dns resolution issue.<o:p></o:p></p><p class=MsoNormal>In the past I remember that Docker was updating the hosts file with the relevant names but it’s not working the same way now.<o:p></o:p></p><p class=MsoNormal>Currently Docker is using a local network dns service which is being accessed via 127.0.0.53.<o:p></o:p></p><p class=MsoNormal>From I remember Squid is resolving the icap service name only at startup or reload.<o:p></o:p></p><p class=MsoNormal>Lately Alex published a testable patch that might fix specific issues with icap services which are resolved by dns. ( sorry I don’t remember the bug report)<o:p></o:p></p><p class=MsoNormal>I assume you can try to test this patch first.<o:p></o:p></p><p class=MsoNormal>If these services are static to some degree you might be able to create a script that updates the hosts file and reload squid on each change.<o:p></o:p></p><p class=MsoNormal>When using the hosts file it’s possible that some issues will disappear.<o:p></o:p></p><p class=MsoNormal><br>There is also another possibility which is a malformed ICAP response or wrong sessions handling which cause this issue.<o:p></o:p></p><p class=MsoNormal>You might be able to use tcpdump from either the host or the container side to capture traffic when these goes down.<o:p></o:p></p><p class=MsoNormal>Depends on your preference of debug level you might even be able to debug specific debug_options like for ICAP services<br>and/or requests to the degree you might be able to see what happens on the basic level of the ICAP encapsulation.<o:p></o:p></p><p class=MsoNormal>If you really need help with a diagnosis and a solution you might be able to use Alex and the measurement factory.<br><br><o:p></o:p></p><p class=MsoNormal>All The Bests,<o:p></o:p></p><p class=MsoNormal>Eliezer<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b>From:</b> squid-users <squid-users-bounces@lists.squid-cache.org> <b>On Behalf Of </b>roie rachamim<br><b>Sent:</b> Monday, April 12, 2021 12:54 PM<br><b>To:</b> squid-users@lists.squid-cache.org<br><b>Subject:</b> [squid-users] All Adaptation ICAPs go down at the same time<o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal>Hi,<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt'>Our setup includes squid that runs in docker container with several ICAP servers in additional containers.<o:p></o:p></p></div><div><p class=MsoNormal>From time to time we see in cache.log the following messages:<br>2021/04/12 00:22:39| optional ICAP service is down after an options fetch failure: icap://icap1.proxy:14590/censor [down,!opt]<br>2021/04/12 00:22:39| optional ICAP service is down after an options fetch failure: icap://icap2.proxy:1344/request [down,!opt]<br>2021/04/12 00:22:39| optional ICAP service is down after an options fetch failure: icap://icap3.proxy:14590/response [down,!opt]<br><br>2021/04/12 06:10:45| optional ICAP service is down after an options fetch failure: icap://icap1.proxy:14590/censor [down,!opt]<br>2021/04/12 06:10:45| optional ICAP service is down after an options fetch failure: icap://icap2.proxy:1344/request [down,!opt]<br>2021/04/12 06:10:45| optional ICAP service is down after an options fetch failure: icap://icap3.proxy:14590/response [down,!opt]<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt'>We're trying to understand why it happens to all ICAPs at once. This happens in 4.14 and in 5.0.4<o:p></o:p></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt'>Any thoughts about what might cause this ?<o:p></o:p></p></div><div><p class=MsoNormal>Many Thanks,<o:p></o:p></p></div><div><p class=MsoNormal>Roie<o:p></o:p></p></div></div></div></body></html>