[squid-users] https full url

[Alex Rousskov]

On 01/16/2016 10:16 AM, xxiao8 wrote:

> I understand the step1/2/3 for sslbump and Squid extracts the domain
> name at various steps(SNI, client hello, server hello,etc) and there is
> no guarantee which step will get what I want.

The stages I was talking about are not directly related to SslBump
steps. They are a higher-level concept.


> Assuming the ssl is fully bumped(not spliced), that means their http
> request-line(the full URL) and http header(e.g. referer) are now
> available in clear text inside Squid,

Yes, a "bumped" HTTPS transaction is very similar to a plain text HTTP
transaction inside Squid. Most of the confusion usually happens on the
way to that blissful stage, not during that stage.


> my question is that, can Squid now
> "extract" them and let me use them via directives, just like I use
> url_regex/urlpath_regex/referer_regex when http mode is used.

Yes. The vast majority of Squid code does not know that the transaction
was bumped.


> I tested external_acl_type and the URI/PATH were empty for _bumped_ ssl
> traffic too.

That means you have not tested what you think you have tested. To use an
analogy: If the test tells us that 2+2=5, then there is something wrong
with the test itself, not arithmetic.

I do not know what is wrong with your test, but I suspect that you are
looking at stage #2 info while thinking that you are looking at stage #4
info. This is just a guess though!


> Basically I'm trying to see how to get the http-header info from a
> bumped ssl connection and use them directly inside squid.conf(including
> external acl), otherwise icap/ecap is unavoidable for bumped ssl http
> header analysis.

There is nothing special about eCAP/ICAP as far as access to transaction
meta information. eCAP/ICAP services can access the same transaction
meta data as Squid ACLs can. You think that eCAP/ICAP services can
access more info than ACLs, but that is not true.

eCAP and ICAP services have certain advantages over ACLs (e.g., they
have access to message bodies), but those advantages are unrelated to
this thread AFAICT.


Alex.


> On 01/16/2016 10:06 AM, Alex Rousskov wrote:
>> On 01/15/2016 07:52 PM, xxiao8 wrote:
>>> Just found out ssl::server_name_regex that should cover url_regex, for
>>> urlpath_regex and referer_regex I think I can not get them for
>>> https/sslbump, to get them an icap/ecap has to be used to read the
>>> decrypted content at the moment, will squid plan to provide directives
>>> similar to urlpath_regex/refere_regex for-sslbump in the future?
>>
>> Your understanding of how HTTPS transactions work still does not match
>> the reality, resulting in confusion about applicability of various Squid
>> features, especially ACLs.
>>
>> One way to improve your understanding of reality is to stop thinking
>> about HTTPS transaction as a single request/response pair. *Roughly
>> speaking*, an HTTPS transaction is a sequence of several stages:
>>
>> 1. TCP connection establishment.
>> 2. CONNECT transaction(s) that are HTTP transactions with limited info.
>> 3. SSL connection establishment (and bumping).
>> 4. Bumped HTTPS transaction(s) treated as regular HTTP transactions.
>>
>> Most HTTP-focused ACLs like url_regex are meant for regular HTTP
>> transactions (stage #4). They are of limited use for CONNECT
>> transactions (stage #2) and usually do not work during stages #1 and #3.
>>
>> ICAP/eCAP adaptation may be applied during stage #2 (with limited
>> information) and stage #4.
>>
>> The statement "referer_regex does not work for https/sslbump" is neither
>> true nor false. Thus, you are getting conflicting responses and are
>> likely to create poorly working configurations until you realize that
>> "referer_regex" works well at stage #4 but may also be applied at stage
>> #2 where it does not work well (not enough information in the CONNECT
>> requests) or at stages #1 and #3 where it should not be used at all.
>>
>>
>> ACL is just a "named condition" or a "function". You can type an ACL
>> name X in any squid.conf directives that use ACLs. That does not mean
>> that ACL X works with all directives at all times. Someday, Squid or
>> some squid.conf lint tool would warn about ACLs used in wrong places.
>> For now, you have to understand what stage-dependent information the
>> directive has access to and avoid ACLs that require information
>> unavailable to the directive at the current processing stage.
>>
>>
>> Good luck,
>>
>> Alex.
>> P.S. For intercepted connections, CONNECT transactions in stage #2 are
>> faked by Squid. For forward-proxied connections, they are real.
>>
> 
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users