[squid-users] Passing Proxy Protocol Headers to external ACL

Sun Jun 20 06:55:52 UTC 2021

As far as I understand the ecap service should be called for each step:

  *   In step1 there is no SNI, and X-PMeta-Splice should be set in the ecap to 'no'.
  *   In step2 there should be an SNI, and X-PMeta-Splice should be set to 'yes', and the classifyRequest acl should return true, causing the request to be spliced.

In the debug log however, I see prints saying that it was decided to bump the request, suggesting that step2 is not working as expected.
Please advise how to proceed?
________________________________
From: Alex Rousskov <rousskov at measurement-factory.com>
Sent: Thursday, June 17, 2021 18:41
To: Frida Safran <fsafran at proofpoint.com>; squid-users at lists.squid-cache.org <squid-users at lists.squid-cache.org>
Subject: Re: [squid-users] Passing Proxy Protocol Headers to external ACL

On 6/17/21 1:55 AM, Frida Safran wrote:

> I see that the issue stems from that the %ssl::sni is not getting passed
> correctly (evaluated as "-") for CONNECT requests when passed via:
>
> adaptation_meta X-PMeta-SNI "%ssl::sni"
>
> Why could that be? I do see that in the access.log the %ssl::sni is
> logged correctly on CONNECT entries.

Perhaps it is because SNI is never available during SslBump step1? It is
not clear (to me) whether you are talking about step1 or step2
adaptation_meta evaluation. Also, there could be Squid bugs that result
in missing adaptation_meta re-evaluations during step2 -- I have not
checked.

> I have worked around this by passing the "%>rd" instead - are these 2
> values equivalent in all cases?

No, they are not.

1. SNI should always be a domain name. CONNECT request URI may be an IP.

1.1 A true CONNECT request may use an IP address -- client choice.

1.2 A faked during step1 CONNECT request uses an IP address.
   This faking always happens when you use SslBump on https_port.

2. SNI may differ from the domain name in the CONNECT request.
  For example, the CONNECT request may be for example.com while
  SNI may say something more specific like service.example.com.
  Some SNI domains are obfuscated/encrypted.

3. SNI may be missing in TLS ClientHello.
   CONNECT requests always have a URI.

HTH,

Alex.

> ------------------------------------------------------------------------
> *From:* Alex Rousskov <rousskov at measurement-factory.com>
> *Sent:* Wednesday, June 16, 2021 17:38
> *To:* Frida Safran <fsafran at proofpoint.com>;
> squid-users at lists.squid-cache.org <squid-users at lists.squid-cache.org>
> *Subject:* Re: [squid-users] Passing Proxy Protocol Headers to external ACL
>
> On 6/16/21 4:15 AM, Frida Safran wrote:
>
>> I see that the configured acl is not called for all requests.
>
> classifyRequest should be evaluated for all CONNECT requests that get to
> SslBump step2. That should be all CONNECT requests except for (rare)
> cases where Squid cannot peek at the TLS ClientHello during step1 and,
> hence, bumps the client connection without reaching step2.
>
> Whether classifyRequest _matches_ when evaluated is a different story,
> of course.
>
>> I do see that the ecap is called, it sets the option X-PMeta-Splice
>> via visitEachOption correctly, but the classifyRequest note acl is not
>> evaluated, and the request is not spliced.
>> However, I do see that this acl is evaluated for some other requests,
>> but only when the X-PMeta-Bypass is set to: 'no'.
>> This config worked as expected when using an external acl type for the
>> same request, but i also saw the same issue when using note when i tried
>> to use it to classify the external acl.
>> Is there perhaps a bug/misconfiguration with the note acl?
>
> Sorry, I do not have enough information to answer that question. Try
> sharing an ALL,9 cache.log that reproduces the problem using a single
> transaction:
> https://urldefense.com/v3/__https://wiki.squid-cache.org/SquidFaq/BugReporting*Debugging_a_single_transaction__;Iw!!ORgEfCBsr282Fw!9mAZmYL4yTa1_p9UoNH3-uc5baXXKzSf1Zi9SkjsnCkMq_JCg50KhEXmwGcowILvtw$
> <https://urldefense.com/v3/__https://wiki.squid-cache.org/SquidFaq/BugReporting*Debugging_a_single_transaction__;Iw!!ORgEfCBsr282Fw!9mAZmYL4yTa1_p9UoNH3-uc5baXXKzSf1Zi9SkjsnCkMq_JCg50KhEXmwGcowILvtw$>
>
>
>
>> The current config is:
>>
>> acl classifyRequest note -m X-PMeta-Bypass yes
>>
>> acl step1 at_step SslBump1
>> acl step2 at_step SslBump2
>>
>> ssl_bump peek step1
>> ssl_bump splice step2 classifyRequest
>> ssl_bump stare all
>> ssl_bump bump all
>
>> adaptation_meta X-PMeta-Bypass "%adapt::<last_h{X-PMeta-Splice}"
>
> This adaptation_meta configuration should pass X-PMeta-Bypass
> meta-header field to the adaptation service, with the field value set to
> the value of the value of X-PMeta-Splice meta-header field received by
> Squid from the previously applied adaption service (within the same
> master transaction). Is that what you want?
>
>
> HTH,
>
> Alex.
>
>
>
>> ------------------------------------------------------------------------
>> *From:* Alex Rousskov <rousskov at measurement-factory.com>
>> *Sent:* Tuesday, June 15, 2021 21:35
>> *To:* Frida Safran <fsafran at proofpoint.com>;
>> squid-users at lists.squid-cache.org <squid-users at lists.squid-cache.org>
>> *Subject:* Re: [squid-users] Passing Proxy Protocol Headers to external ACL
>>
>> On 6/15/21 4:18 AM, Frida Safran wrote:
>>
>>> In addition to using an external acl to annotate connections and decide
>>> whether splice/bump, I would like to try using an ecap service to
>>> achieve this.
>>> I would like to create an acl using info from the ecap service, and
>>> bump/splice using the following configuration:
>>>
>>> acl classifyRequest note splice yes
>>>
>>> acl step1 at_step SslBump1
>>> acl step2 at_step SslBump2
>>>
>>> ssl_bump peek step1
>>> ssl_bump splice step2 classifyRequest
>>> ssl_bump stare all
>>> ssl_bump bump all
>>>
>>>
>>> If I set a custom option from within the ecap service, can i use that
>>> via the above note acl?
>>
>> I believe that should work in principle, provided the eCAP service is
>> consulted before Squid starts evaluating "ssl_bump splice step2
>> classifyRequest". If you do not restrict what requests your eCAP REQMOD
>> service gets, then it should be consulted during step1 (at least). The
>> logic is the same for eCAP and ICAP here -- Squid "primary" code does
>> not know the difference between those two adaptation interfaces.
>>
>>
>>> Can i set a custom option without setting it in
>>> 'adaptation_masterx_shared_names'
>>
>> Yes, you should be able to. The adaptation_masterx_shared_names
>> directive is unrelated to setting transaction annotations IIRC. It is
>> about sharing metadata across adaptation services.
>>
>>
>>> Or, should I instead use:
>>>
>>> acl classifyRequest note %adapt::<last_h{splice} yes
>>
>> This option cannot work AFAICT because the "note" ACL requires a
>> constant (i.e. known at configuration time) annotation name. Squid will
>> not substitute %adapt::<last_h{splice} with anything in this context.
>>
>>
>> HTH,
>>
>> Alex.
>>
>>
>>> ------------------------------------------------------------------------
>>> *From:* Alex Rousskov <rousskov at measurement-factory.com>
>>> *Sent:* Monday, June 14, 2021 16:24
>>> *To:* squid-users at lists.squid-cache.org <squid-users at lists.squid-cache.org>
>>> *Cc:* Frida Safran <fsafran at proofpoint.com>
>>> *Subject:* Re: [squid-users] Passing Proxy Protocol Headers to external ACL
>>>
>>> On 6/14/21 2:29 AM, Frida Safran wrote:
>>>
>>>> Regarding proxy_protocol - is there a known patch for v4 I could use by
>>>> any chance?
>>>
>>> I am not aware of any such patches. The changes were significant, fixing
>>> many PROXY protocol handling bugs. Virtually anything can be backported,
>>> but it would be a large effort with noticeable stability risks and
>>> long-term maintenance overheads. Preparing for a v5 upgrade may be a
>>> better strategy in this particular case.
>>>
>>>
>>>> Regarding icap, I suppose the acl is getting evaluated before the icap
>>>> and that is why they aren't available:
>>>
>>>> acl classifyRequest external TransactionClassificator
>>>> ssl_bump peek step1
>>>> ssl_bump splice step2 classifyRequest
>>>> ssl_bump stare all
>>>> ssl_bump bump all
>>>
>>> According to [1], the above configuration should result in two ICAP
>>> REQMOD requests (if configured) before classifyRequest is consulted
>>> during step2. I am aware of SslBump bugs in that area, but I would
>>> expect at least one ICAP REQMOD request anyway. The requests
>>> existence/timing should be easy to confirm using cache.log with
>>> debug_options set to at least "ALL,3 82,9 93,9" and/or a logging or
>>> pausing external ACL script in combination with an icap_log (to compare
>>> logged timestamps).
>>>
>>> [1]
>>> https://urldefense.com/v3/__https://wiki.squid-cache.org/Features/SslPeekAndSplice__;!!ORgEfCBsr282Fw!6SDlCpq2n2kV1WuiNQ7focWt2YMDj-Xs9aEp29dC32pwZUHSLIkrD7dnojPghFBhQQ$
> <https://urldefense.com/v3/__https://wiki.squid-cache.org/Features/SslPeekAndSplice__;!!ORgEfCBsr282Fw!6SDlCpq2n2kV1WuiNQ7focWt2YMDj-Xs9aEp29dC32pwZUHSLIkrD7dnojPghFBhQQ$>
>> <https://urldefense.com/v3/__https://wiki.squid-cache.org/Features/SslPeekAndSplice__;!!ORgEfCBsr282Fw!6SDlCpq2n2kV1WuiNQ7focWt2YMDj-Xs9aEp29dC32pwZUHSLIkrD7dnojPghFBhQQ$
> <https://urldefense.com/v3/__https://wiki.squid-cache.org/Features/SslPeekAndSplice__;!!ORgEfCBsr282Fw!6SDlCpq2n2kV1WuiNQ7focWt2YMDj-Xs9aEp29dC32pwZUHSLIkrD7dnojPghFBhQQ$>>
>>> <https://urldefense.com/v3/__https://wiki.squid-cache.org/Features/SslPeekAndSplice__;!!ORgEfCBsr282Fw!6SDlCpq2n2kV1WuiNQ7focWt2YMDj-Xs9aEp29dC32pwZUHSLIkrD7dnojPghFBhQQ$
>> <https://urldefense.com/v3/__https://wiki.squid-cache.org/Features/SslPeekAndSplice__;!!ORgEfCBsr282Fw!6SDlCpq2n2kV1WuiNQ7focWt2YMDj-Xs9aEp29dC32pwZUHSLIkrD7dnojPghFBhQQ$
> <https://urldefense.com/v3/__https://wiki.squid-cache.org/Features/SslPeekAndSplice__;!!ORgEfCBsr282Fw!6SDlCpq2n2kV1WuiNQ7focWt2YMDj-Xs9aEp29dC32pwZUHSLIkrD7dnojPghFBhQQ$>>>
>>>
>>>
>>>
>>> HTH,
>>>
>>> Alex.
>>>
>>>
>>>> ------------------------------------------------------------------------
>>>> *From:* Alex Rousskov <rousskov at measurement-factory.com>
>>>> *Sent:* Sunday, June 13, 2021 17:46
>>>> *To:* squid-users at lists.squid-cache.org <squid-users at lists.squid-cache.org>
>>>> *Cc:* Frida Safran <fsafran at proofpoint.com>
>>>> *Subject:* Re: [squid-users] Passing Proxy Protocol Headers to external ACL
>>>>
>>>> On 6/13/21 7:31 AM, Frida Safran wrote:
>>>>
>>>>>  1. Is it possible to pass proxy protocol headers to an external acl as
>>>>>     part of the format?
>>>>
>>>> It should be possible. Use %proxy_protocol::>h logformat %code in your
>>>> external_acl_type FORMAT configuration. We added that support to Squid
>>>> v5. Not available in the official v4.
>>>>
>>>>
>>>>>  2. Is it possible to pass all/specific icap headers to an external acl?
>>>>>     I have been trying to use %icap::>h to pass all the icap headers to
>>>>>     an external acl, but it resolves to "-"
>>>>
>>>> It should be possible if your external ACL is evaluated _after_ the
>>>> corresponding ICAP headers are received, but I would not be surprised if
>>>> there are bugs in this area -- the ICAP headers may be available but not
>>>>  provided to the ACL evaluation code. Which squid.conf directive is
>>>> triggering your external ACL evaluation in this use case?
>>>>
>>>>
>>>> HTH,
>>>>
>>>> Alex.
>>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20210620/7359613b/attachment-0001.htm>