<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;">As far as I understand the ecap service should be called for each step:</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<ul>
<li><span><span style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;">In step1 there is no SNI, and </span><span style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt;">X-PMeta-Splice should be set in the ecap to 'no'.</span></span></li><li>In step2 there should be an SNI, and <span style="color: rgb(0, 0, 0); font-size: 12pt; font-family: Calibri, Helvetica, sans-serif;">X-PMeta-Splice should be set to 'yes', and the classifyRequest acl should return true, causing the request to be spliced.</span></li></ul>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgb(255, 255, 255); display: inline !important;">In the debug log however, I see prints saying that it was decided to bump the request, suggesting that step2 is not
working as expected.</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Please advise how to proceed?</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Alex Rousskov <rousskov@measurement-factory.com><br>
<b>Sent:</b> Thursday, June 17, 2021 18:41<br>
<b>To:</b> Frida Safran <fsafran@proofpoint.com>; squid-users@lists.squid-cache.org <squid-users@lists.squid-cache.org><br>
<b>Subject:</b> Re: [squid-users] Passing Proxy Protocol Headers to external ACL</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">On 6/17/21 1:55 AM, Frida Safran wrote:<br>
<br>
> I see that the issue stems from that the %ssl::sni is not getting passed<br>
> correctly (evaluated as "-") for CONNECT requests when passed via:<br>
> <br>
> adaptation_meta X-PMeta-SNI "%ssl::sni"<br>
> <br>
> Why could that be? I do see that in the access.log the %ssl::sni is<br>
> logged correctly on CONNECT entries.<br>
<br>
<br>
Perhaps it is because SNI is never available during SslBump step1? It is<br>
not clear (to me) whether you are talking about step1 or step2<br>
adaptation_meta evaluation. Also, there could be Squid bugs that result<br>
in missing adaptation_meta re-evaluations during step2 -- I have not<br>
checked.<br>
<br>
<br>
> I have worked around this by passing the "%>rd" instead - are these 2<br>
> values equivalent in all cases?<br>
<br>
No, they are not.<br>
<br>
1. SNI should always be a domain name. CONNECT request URI may be an IP.<br>
<br>
1.1 A true CONNECT request may use an IP address -- client choice.<br>
<br>
1.2 A faked during step1 CONNECT request uses an IP address.<br>
This faking always happens when you use SslBump on https_port.<br>
<br>
2. SNI may differ from the domain name in the CONNECT request.<br>
For example, the CONNECT request may be for example.com while<br>
SNI may say something more specific like service.example.com.<br>
Some SNI domains are obfuscated/encrypted.<br>
<br>
3. SNI may be missing in TLS ClientHello.<br>
CONNECT requests always have a URI.<br>
<br>
HTH,<br>
<br>
Alex.<br>
<br>
> ------------------------------------------------------------------------<br>
> *From:* Alex Rousskov <rousskov@measurement-factory.com><br>
> *Sent:* Wednesday, June 16, 2021 17:38<br>
> *To:* Frida Safran <fsafran@proofpoint.com>;<br>
> squid-users@lists.squid-cache.org <squid-users@lists.squid-cache.org><br>
> *Subject:* Re: [squid-users] Passing Proxy Protocol Headers to external ACL<br>
> <br>
> On 6/16/21 4:15 AM, Frida Safran wrote:<br>
> <br>
>> I see that the configured acl is not called for all requests.<br>
> <br>
> classifyRequest should be evaluated for all CONNECT requests that get to<br>
> SslBump step2. That should be all CONNECT requests except for (rare)<br>
> cases where Squid cannot peek at the TLS ClientHello during step1 and,<br>
> hence, bumps the client connection without reaching step2.<br>
> <br>
> Whether classifyRequest _matches_ when evaluated is a different story,<br>
> of course.<br>
> <br>
>> I do see that the ecap is called, it sets the option X-PMeta-Splice<br>
>> via visitEachOption correctly, but the classifyRequest note acl is not<br>
>> evaluated, and the request is not spliced.<br>
>> However, I do see that this acl is evaluated for some other requests,<br>
>> but only when the X-PMeta-Bypass is set to: 'no'.<br>
>> This config worked as expected when using an external acl type for the<br>
>> same request, but i also saw the same issue when using note when i tried<br>
>> to use it to classify the external acl.<br>
>> Is there perhaps a bug/misconfiguration with the note acl?<br>
> <br>
> Sorry, I do not have enough information to answer that question. Try<br>
> sharing an ALL,9 cache.log that reproduces the problem using a single<br>
> transaction:<br>
> <a href="https://urldefense.com/v3/__https://wiki.squid-cache.org/SquidFaq/BugReporting*Debugging_a_single_transaction__;Iw!!ORgEfCBsr282Fw!9mAZmYL4yTa1_p9UoNH3-uc5baXXKzSf1Zi9SkjsnCkMq_JCg50KhEXmwGcowILvtw$">
https://urldefense.com/v3/__https://wiki.squid-cache.org/SquidFaq/BugReporting*Debugging_a_single_transaction__;Iw!!ORgEfCBsr282Fw!9mAZmYL4yTa1_p9UoNH3-uc5baXXKzSf1Zi9SkjsnCkMq_JCg50KhEXmwGcowILvtw$</a><br>
> <<a href="https://urldefense.com/v3/__https://wiki.squid-cache.org/SquidFaq/BugReporting*Debugging_a_single_transaction__;Iw!!ORgEfCBsr282Fw!9mAZmYL4yTa1_p9UoNH3-uc5baXXKzSf1Zi9SkjsnCkMq_JCg50KhEXmwGcowILvtw$">https://urldefense.com/v3/__https://wiki.squid-cache.org/SquidFaq/BugReporting*Debugging_a_single_transaction__;Iw!!ORgEfCBsr282Fw!9mAZmYL4yTa1_p9UoNH3-uc5baXXKzSf1Zi9SkjsnCkMq_JCg50KhEXmwGcowILvtw$</a>><br>
> <br>
> <br>
> <br>
>> The current config is:<br>
>> <br>
>> acl classifyRequest note -m X-PMeta-Bypass yes<br>
>> <br>
>> acl step1 at_step SslBump1<br>
>> acl step2 at_step SslBump2<br>
>> <br>
>> ssl_bump peek step1<br>
>> ssl_bump splice step2 classifyRequest<br>
>> ssl_bump stare all<br>
>> ssl_bump bump all<br>
> <br>
>> adaptation_meta X-PMeta-Bypass "%adapt::<last_h{X-PMeta-Splice}"<br>
> <br>
> This adaptation_meta configuration should pass X-PMeta-Bypass<br>
> meta-header field to the adaptation service, with the field value set to<br>
> the value of the value of X-PMeta-Splice meta-header field received by<br>
> Squid from the previously applied adaption service (within the same<br>
> master transaction). Is that what you want?<br>
> <br>
> <br>
> HTH,<br>
> <br>
> Alex.<br>
> <br>
> <br>
> <br>
>> ------------------------------------------------------------------------<br>
>> *From:* Alex Rousskov <rousskov@measurement-factory.com><br>
>> *Sent:* Tuesday, June 15, 2021 21:35<br>
>> *To:* Frida Safran <fsafran@proofpoint.com>;<br>
>> squid-users@lists.squid-cache.org <squid-users@lists.squid-cache.org><br>
>> *Subject:* Re: [squid-users] Passing Proxy Protocol Headers to external ACL<br>
>> <br>
>> On 6/15/21 4:18 AM, Frida Safran wrote:<br>
>> <br>
>>> In addition to using an external acl to annotate connections and decide<br>
>>> whether splice/bump, I would like to try using an ecap service to<br>
>>> achieve this.<br>
>>> I would like to create an acl using info from the ecap service, and<br>
>>> bump/splice using the following configuration:<br>
>>> <br>
>>> acl classifyRequest note splice yes<br>
>>> <br>
>>> acl step1 at_step SslBump1<br>
>>> acl step2 at_step SslBump2<br>
>>> <br>
>>> ssl_bump peek step1<br>
>>> ssl_bump splice step2 classifyRequest<br>
>>> ssl_bump stare all<br>
>>> ssl_bump bump all<br>
>>> <br>
>>> <br>
>>> If I set a custom option from within the ecap service, can i use that<br>
>>> via the above note acl?<br>
>> <br>
>> I believe that should work in principle, provided the eCAP service is<br>
>> consulted before Squid starts evaluating "ssl_bump splice step2<br>
>> classifyRequest". If you do not restrict what requests your eCAP REQMOD<br>
>> service gets, then it should be consulted during step1 (at least). The<br>
>> logic is the same for eCAP and ICAP here -- Squid "primary" code does<br>
>> not know the difference between those two adaptation interfaces.<br>
>> <br>
>> <br>
>>> Can i set a custom option without setting it in<br>
>>> 'adaptation_masterx_shared_names'<br>
>> <br>
>> Yes, you should be able to. The adaptation_masterx_shared_names<br>
>> directive is unrelated to setting transaction annotations IIRC. It is<br>
>> about sharing metadata across adaptation services.<br>
>> <br>
>> <br>
>>> Or, should I instead use:<br>
>>> <br>
>>> acl classifyRequest note %adapt::<last_h{splice} yes<br>
>> <br>
>> This option cannot work AFAICT because the "note" ACL requires a<br>
>> constant (i.e. known at configuration time) annotation name. Squid will<br>
>> not substitute %adapt::<last_h{splice} with anything in this context.<br>
>> <br>
>> <br>
>> HTH,<br>
>> <br>
>> Alex.<br>
>> <br>
>> <br>
>>> ------------------------------------------------------------------------<br>
>>> *From:* Alex Rousskov <rousskov@measurement-factory.com><br>
>>> *Sent:* Monday, June 14, 2021 16:24<br>
>>> *To:* squid-users@lists.squid-cache.org <squid-users@lists.squid-cache.org><br>
>>> *Cc:* Frida Safran <fsafran@proofpoint.com><br>
>>> *Subject:* Re: [squid-users] Passing Proxy Protocol Headers to external ACL<br>
>>> <br>
>>> On 6/14/21 2:29 AM, Frida Safran wrote:<br>
>>> <br>
>>>> Regarding proxy_protocol - is there a known patch for v4 I could use by<br>
>>>> any chance?<br>
>>> <br>
>>> I am not aware of any such patches. The changes were significant, fixing<br>
>>> many PROXY protocol handling bugs. Virtually anything can be backported,<br>
>>> but it would be a large effort with noticeable stability risks and<br>
>>> long-term maintenance overheads. Preparing for a v5 upgrade may be a<br>
>>> better strategy in this particular case.<br>
>>> <br>
>>> <br>
>>>> Regarding icap, I suppose the acl is getting evaluated before the icap<br>
>>>> and that is why they aren't available:<br>
>>> <br>
>>>> acl classifyRequest external TransactionClassificator<br>
>>>> ssl_bump peek step1<br>
>>>> ssl_bump splice step2 classifyRequest<br>
>>>> ssl_bump stare all<br>
>>>> ssl_bump bump all<br>
>>> <br>
>>> According to [1], the above configuration should result in two ICAP<br>
>>> REQMOD requests (if configured) before classifyRequest is consulted<br>
>>> during step2. I am aware of SslBump bugs in that area, but I would<br>
>>> expect at least one ICAP REQMOD request anyway. The requests<br>
>>> existence/timing should be easy to confirm using cache.log with<br>
>>> debug_options set to at least "ALL,3 82,9 93,9" and/or a logging or<br>
>>> pausing external ACL script in combination with an icap_log (to compare<br>
>>> logged timestamps).<br>
>>> <br>
>>> [1]<br>
>>> <a href="https://urldefense.com/v3/__https://wiki.squid-cache.org/Features/SslPeekAndSplice__;!!ORgEfCBsr282Fw!6SDlCpq2n2kV1WuiNQ7focWt2YMDj-Xs9aEp29dC32pwZUHSLIkrD7dnojPghFBhQQ$">
https://urldefense.com/v3/__https://wiki.squid-cache.org/Features/SslPeekAndSplice__;!!ORgEfCBsr282Fw!6SDlCpq2n2kV1WuiNQ7focWt2YMDj-Xs9aEp29dC32pwZUHSLIkrD7dnojPghFBhQQ$</a><br>
> <<a href="https://urldefense.com/v3/__https://wiki.squid-cache.org/Features/SslPeekAndSplice__;!!ORgEfCBsr282Fw!6SDlCpq2n2kV1WuiNQ7focWt2YMDj-Xs9aEp29dC32pwZUHSLIkrD7dnojPghFBhQQ$">https://urldefense.com/v3/__https://wiki.squid-cache.org/Features/SslPeekAndSplice__;!!ORgEfCBsr282Fw!6SDlCpq2n2kV1WuiNQ7focWt2YMDj-Xs9aEp29dC32pwZUHSLIkrD7dnojPghFBhQQ$</a>><br>
>> <<a href=""></a>https://urldefense.com/v3/__https://wiki.squid-cache.org/Features/SslPeekAndSplice__;!!ORgEfCBsr282Fw!6SDlCpq2n2kV1WuiNQ7focWt2YMDj-Xs9aEp29dC32pwZUHSLIkrD7dnojPghFBhQQ$<br>
> <<a href="https://urldefense.com/v3/__https://wiki.squid-cache.org/Features/SslPeekAndSplice__;!!ORgEfCBsr282Fw!6SDlCpq2n2kV1WuiNQ7focWt2YMDj-Xs9aEp29dC32pwZUHSLIkrD7dnojPghFBhQQ$">https://urldefense.com/v3/__https://wiki.squid-cache.org/Features/SslPeekAndSplice__;!!ORgEfCBsr282Fw!6SDlCpq2n2kV1WuiNQ7focWt2YMDj-Xs9aEp29dC32pwZUHSLIkrD7dnojPghFBhQQ$</a>>><br>
>>> <<a href=""></a>https://urldefense.com/v3/__https://wiki.squid-cache.org/Features/SslPeekAndSplice__;!!ORgEfCBsr282Fw!6SDlCpq2n2kV1WuiNQ7focWt2YMDj-Xs9aEp29dC32pwZUHSLIkrD7dnojPghFBhQQ$<br>
>> <<a href=""></a>https://urldefense.com/v3/__https://wiki.squid-cache.org/Features/SslPeekAndSplice__;!!ORgEfCBsr282Fw!6SDlCpq2n2kV1WuiNQ7focWt2YMDj-Xs9aEp29dC32pwZUHSLIkrD7dnojPghFBhQQ$<br>
> <<a href="https://urldefense.com/v3/__https://wiki.squid-cache.org/Features/SslPeekAndSplice__;!!ORgEfCBsr282Fw!6SDlCpq2n2kV1WuiNQ7focWt2YMDj-Xs9aEp29dC32pwZUHSLIkrD7dnojPghFBhQQ$">https://urldefense.com/v3/__https://wiki.squid-cache.org/Features/SslPeekAndSplice__;!!ORgEfCBsr282Fw!6SDlCpq2n2kV1WuiNQ7focWt2YMDj-Xs9aEp29dC32pwZUHSLIkrD7dnojPghFBhQQ$</a>>>><br>
>>> <br>
>>> <br>
>>> <br>
>>> HTH,<br>
>>> <br>
>>> Alex.<br>
>>> <br>
>>> <br>
>>>> ------------------------------------------------------------------------<br>
>>>> *From:* Alex Rousskov <rousskov@measurement-factory.com><br>
>>>> *Sent:* Sunday, June 13, 2021 17:46<br>
>>>> *To:* squid-users@lists.squid-cache.org <squid-users@lists.squid-cache.org><br>
>>>> *Cc:* Frida Safran <fsafran@proofpoint.com><br>
>>>> *Subject:* Re: [squid-users] Passing Proxy Protocol Headers to external ACL<br>
>>>> <br>
>>>> On 6/13/21 7:31 AM, Frida Safran wrote:<br>
>>>> <br>
>>>>> 1. Is it possible to pass proxy protocol headers to an external acl as<br>
>>>>> part of the format?<br>
>>>> <br>
>>>> It should be possible. Use %proxy_protocol::>h logformat %code in your<br>
>>>> external_acl_type FORMAT configuration. We added that support to Squid<br>
>>>> v5. Not available in the official v4.<br>
>>>> <br>
>>>> <br>
>>>>> 2. Is it possible to pass all/specific icap headers to an external acl?<br>
>>>>> I have been trying to use %icap::>h to pass all the icap headers to<br>
>>>>> an external acl, but it resolves to "-"<br>
>>>> <br>
>>>> It should be possible if your external ACL is evaluated _after_ the<br>
>>>> corresponding ICAP headers are received, but I would not be surprised if<br>
>>>> there are bugs in this area -- the ICAP headers may be available but not<br>
>>>> provided to the ACL evaluation code. Which squid.conf directive is<br>
>>>> triggering your external ACL evaluation in this use case?<br>
>>>> <br>
>>>> <br>
>>>> HTH,<br>
>>>> <br>
>>>> Alex.<br>
>>> <br>
>> <br>
> <br>
<br>
</div>
</span></font></div>
</body>
</html>