[squid-users] Rewriting HTTP to HTTPS for generic package proxy

Sun Jul 14 21:02:43 UTC 2024

I did some more debugging and I think that I have found the cause why the issue occurs in case (A). As Alex already explained, in case (A) the child proxy forwards the rewritten request e.g. a GET request containing a HTTPS URL, to the parent proxy. Now the parent proxy is in charge to establish a connection with the target server. This action fails with:

TLS code: SQUID_TLS_ERR_CONNECT+GNUTLS_E_FATAL_ALERT_RECEIVED
SSL handshake error (SQUID_TLS_ERR_CONNECT)

The problem is when you use Squid in combination with GnuTLS, the client HELLO message does not contain the server name in the SNI extension. This causes the server to send a FATAL ALERT - Handshake Failure to the client. That is the reason why the parent server cannot establish a secure connection to the target server. I checked the code and it looks promising that Squid with the OpenSSL library will work. In the class support.cc the server name is added when the SNI extension is enabled in the OpenSSL library.

I hope that it is possible for us to switch to the squid-openssl package.

Regards,
Christoph

>-----Ursprüngliche Nachricht-----
>Von: squid-users <squid-users-bounces at lists.squid-cache.org> Im Auftrag von Fiehe,
>Christoph
>Gesendet: Sonntag, 14. Juli 2024 18:59
>An: squid-users at lists.squid-cache.org
>Betreff: Re: [squid-users] Rewriting HTTP to HTTPS for generic package proxy
>
>Hi Alex,
>
>sorry, I have not seen your message, yet. Thank you very much for your helping support.
>
>(A) I will try to find a way to test, how a new Squid build based on OpenSSL behaves under
>those circumstances. It will take some time.
>
>(B) Yes, Squid does nothing wrong, it is a very specific use case. I would prefer to
>utilize a dedicated package caching solution, but there are only a few available. It's a
>pity that Apt-Cacher-NG is in such a bad state. Unfortunately, I lack the required C/C++
>skills to do the code extensions in Squid by myself.
>
>Have a nice weekend.
>
>Regards,
>Christoph
>
>
>>-----Ursprüngliche Nachricht-----
>>Von: squid-users <squid-users-bounces at lists.squid-cache.org> Im Auftrag von Alex Rousskov
>>Gesendet: Donnerstag, 11. Juli 2024 20:27
>>An: squid-users at lists.squid-cache.org
>>Betreff: Re: [squid-users] Rewriting HTTP to HTTPS for generic package proxy
>>
>>On 2024-07-11 13:37, Fiehe, Christoph wrote:
>>> My proxy (the child proxy) already uses the OpenSSL library:
>>
>>Good.
>>
>>
>>> The parent proxy was compiled ... '--with-gnutls'
>>
>>> The GnuTLS exception is thrown at my parent proxy.
>>
>>Thank you for reminding me of that fact; I did not notice or have
>>forgotten about it. I assume you cannot rebuild your parent proxy to use
>>OpenSSL.
>>
>>I see the following choice:
>>
>>A) Continue with the current no-CONNECT setup: Find somebody who can
>>help you get Squid+GnuTLS code path working on the parent proxy. It
>>might be impossible to get this working without making build or
>>configuration changes at the parent proxy. Moreover, please note that
>>your current no-CONNECT setup lacks encryption on the child-parent
>>segment. If that was not intentional, then fixing that will increase
>>TLS-related work for the parent, potentially triggering more problems there.
>>
>>B) Switch to a CONNECT-based setup: Find somebody who can enhance Squid
>>code to establish a CONNECT tunnel through parent proxy when dealing
>>with a GET-https request. Today, Squid will not do that AFAICT[^1].
>>
>>https://wiki.squid-cache.org/SquidFaq/AboutSquid#how-to-add-a-new-squid-feature-enhance-
>>of-fix-something
>>
>>
>>[^1]: AFAICT: Today, there are two primary Squid code conditions for
>>establishing a CONNECT tunnel on a caching code path: Request method is
>>CONNECT or SslBump is in use. Neither matches your GET-https request
>>scenario. Squid current behavior is not "wrong" (as detailed in my
>>earlier email about CONNECT and no-CONNECT scenarios), so, to make these
>>changes official, the author will need to add a configuration option to
>>let admins enable this behavior. The corresponding code changes feel
>>straightforward to me, but I have not studied any details.
>>
>>
>>HTH,
>>
>>Alex.
>>
>>
>>
>>> Unfortunately, I cannot make any changes here. So yes, I trust my parent proxy, but not
>>using a tunnel between child and parent does not seem to work and results in the TLS
>>exception on the parent proxy.
>>>
>>> I have not find a way to tell my child proxy to always setup a tunnel through the
>parent
>>proxy, when the target server talks HTTPS. Do you know, how to achieve that? It would be
>a
>>promising approach.
>>>
>>> Thank you very much help and your patience, Alex.
>>>
>>> Regards,
>>> Christoph
>>>
>>>
>>>> -----Ursprüngliche Nachricht-----
>>>> Von: squid-users <squid-users-bounces at lists.squid-cache.org> Im Auftrag von Alex
>>Rousskov
>>>> Gesendet: Donnerstag, 11. Juli 2024 18:15
>>>> An: squid-users at lists.squid-cache.org
>>>> Betreff: Re: [squid-users] Rewriting HTTP to HTTPS for generic package proxy
>>>>
>>>> On 2024-07-10 16:57, Fiehe, Christoph wrote:
>>>>
>>>>> I am just trying to find something that helps to narrow down the
>>>>> problem. What I want to achieve is, that a client can use HTTP in the
>>>>> LAN, so that Squid can cache distribution packages without making use
>>>>> of SSL intercepting when repos are only accessible via HTTPS.
>>>>
>>>> OK.
>>>>
>>>>
>>>>> In that case the secure connection must start at the proxy and end on
>>>>> the target server with or without any upstream proxies in betweem.
>>>>
>>>> It depends on whether you trust the parent proxy:
>>>>
>>>> If you trust the parent proxy, then you can use two secure connections:
>>>>
>>>> 1.1. child - parent (TLS; no CONNECT)
>>>> 1.2. parent - origin (TLS; no CONNECT)
>>>>
>>>> If you do not trust the parent proxy, then, yes, you will need a tunnel:
>>>>
>>>> 2.1. child - parent (CONNECT)
>>>> 2.2. child - origin (TLS inside the CONNECT tunnel)
>>>>
>>>> N.B. CONNECT request in 2.1 may be plain text (common) or encrypted
>>>> (rare); I am ignoring the difference between those two subcases for now.
>>>>
>>>>
>>>>> We have the following setup:
>>>>>
>>>>> client -> downstream proxy -> upstream proxy -> https://download.docker.com
>>>>>
>>>>> Now let us assume the client wants to retrieve the following resource
>>>> http://download.docker.com/linux/ubuntu/dists/jammy/InRelease from the upstream proxy.
>>>>>
>>>>> The client initiates a HTTP GET request and sends it to the downstream proxy. Now,
>the
>>>> URL gets rewritten.
>>>>
>>>> OK.
>>>>
>>>>
>>>>> It indicates to use a HTTPS connection instead in order to talk to the target server,
>>in
>>>> our case the result is https://download.docker.com/linux/ubuntu/dists/jammy/InRelease.
>>>>
>>>> Yes, but HTTPS scheme does not imply that the child Squid has to use
>>>> CONNECT. There are two possible scenarios detailed above. I do not know
>>>> which of them applies to your use case.
>>>>
>>>>
>>>>> Now comes the critical point: From my understanding – it may be
>>>>> wrongof course - the downstream server now has to send a CONNECT
>>>>> request to the upstream server
>>>>
>>>> Yes, provided the child (downstream) proxy does not trust that parent
>>>> (upstream) proxy. That is scenario 2. Scenario 1 is different.
>>>>
>>>>
>>>>> to advise him to establish a secure connection to the target server.
>>>>
>>>> No, the CONNECT tunnel itself is just a pair of TCP connections. The
>>>> parent proxy "secures" nothing but basic TCP connectivity. It is the
>>>> child proxy that negotiates TLS (over/inside that tunnel) with the
>>>> origin server.
>>>>
>>>>
>>>>> After creation, the downstream proxy can retrieve the resource and
>>>>> send it back to the client via plain HTTP.
>>>>
>>>> Yes.
>>>>
>>>>
>>>>
>>>>> I suppose, that the GnuTLS occurs because of a missing SSL handshake
>>>>> between downstream proxy and download.docker.com.
>>>>
>>>> At this time, I can only say that a TLS negotiation error occurs (while
>>>> child Squid is using the encryption library it probably should not be
>>>> using for this). It is not yet clear to me whether child Squid is
>>>> negotiating with the wrong hop or something goes wrong during
>>>> negotiation with the right hop.
>>>>
>>>> As the next steps, I recommend switching to OpenSSL and, if that alone
>>>> does not help, sharing new errors and determining whether you want to
>>>> use scenario 1 (no CONNECT), scenario 2 (CONNECT), or either (whichever
>>>> works): Do you trust the parent Squid?
>>>>
>>>>
>>>> HTH,
>>>>
>>>> Alex.
>>>>
>>>>
>>>>>> -----Ursprüngliche Nachricht-----
>>>>>> Von: Alex Rousskov <rousskov at measurement-factory.com>
>>>>>> Gesendet: Mittwoch, 10. Juli 2024 22:15
>>>>>> An: squid-users at lists.squid-cache.org
>>>>>> Cc: Fiehe, Christoph <c.fiehe at eurodata.de>
>>>>>> Betreff: AW: [squid-users] Rewriting HTTP to HTTPS for generic package proxy
>>>>>>
>>>>>> On 2024-07-10 15:31, Fiehe, Christoph wrote:
>>>>>>> The problem is that the proxy just forwards the client GET request to the upstream
>>>> proxy
>>>>>>
>>>>>> Why does sending a GET request to the upstream proxy represent a problem
>>>>>> in your use case? I cannot find anything in your prior messages on this
>>>>>> thread that would preclude sending a GET request to the upstream proxy.
>>>>>>
>>>>>>
>>>>>>> but in that case a CONNECT is required.
>>>>>>
>>>>>> Why?
>>>>>>
>>>>>> Please do not interpret my response as implying that this "must send
>>>>>> CONNECT" requirement is wrong (or correct). At this point, I am just
>>>>>> trying to understand what problem(s) you are trying to solve beyond the
>>>>>> one you have originally described.
>>>>>>
>>>>>>
>>>>>> Thank you,
>>>>>>
>>>>>> Alex.
>>>>>
>>>>> _______________________________________________
>>>>> squid-users mailing list
>>>>> squid-users at lists.squid-cache.org
>>>>> https://lists.squid-cache.org/listinfo/squid-users
>>>>
>>>> _______________________________________________
>>>> squid-users mailing list
>>>> squid-users at lists.squid-cache.org
>>>> https://lists.squid-cache.org/listinfo/squid-users
>>> _______________________________________________
>>> squid-users mailing list
>>> squid-users at lists.squid-cache.org
>>> https://lists.squid-cache.org/listinfo/squid-users
>>
>>_______________________________________________
>>squid-users mailing list
>>squid-users at lists.squid-cache.org
>>https://lists.squid-cache.org/listinfo/squid-users
>_______________________________________________
>squid-users mailing list
>squid-users at lists.squid-cache.org
>https://lists.squid-cache.org/listinfo/squid-users