[squid-users] Rewriting HTTP to HTTPS for generic package proxy

Thu Jul 11 17:37:17 UTC 2024

My proxy (the child proxy) already uses the OpenSSL library:

$ squid --version
Squid Cache: Version 6.10
Service Name: squid

This binary uses OpenSSL 3.3.1 4 Jun 2024. configure options:  '--build=x86_64' '--host=x86_64' '--prefix=/usr' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--libexecdir=/usr/lib/squid' '--localstatedir=/var' '--with-logdir=/var/log/squid' '--disable-strict-error-checking' '--disable-arch-native' '--enable-removal-policies=lru,heap' '--enable-auth-digest' '--enable-auth-basic=getpwnam,NCSA,DB,RADIUS' '--enable-basic-auth-helpers=DB' '--enable-external-acl-helpers=file_userip,unix_group,wbinfo_group' '--enable-auth-ntlm=fake' '--enable-auth-negotiate=kerberos,wrapper' '--enable-silent-rules' '--disable-mit' '--enable-heimdal' '--enable-delay-pools' '--enable-openssl' '--enable-ssl-crtd' '--enable-security-cert-generators=file' '--enable-ident-lookups' '--enable-useragent-log' '--enable-cache-digests' '--enable-referer-log' '--enable-async-io' '--enable-truncate' '--enable-arp-acl' '--enable-htcp' '--enable-carp' '--enable-epoll' '--enable-follow-x-forwarded-for' '--enable-storeio=diskd rock' '--enable-ipv6' '--enable-translation' '--enable-snmp' '--disable-dependency-tracking' '--with-large-files' '--with-default-user=squid' '--with-openssl' '--with-pidfile=/var/run/squid/squid.pid' 'build_alias=x86_64' 'host_alias=x86_64' 'CFLAGS=-g0 -O2' 'LDFLAGS=-s' 'CXXFLAGS=-g0 -O2

The parent proxy was compiled with:

squid --version
Squid Cache: Version 6.8
Service Name: squid
Ubuntu linux
configure options:  '--build=x86_64-linux-gnu' '--prefix=/usr' '--includedir=${prefix}/include' '--mandir=${prefix}/share/man' '--infodir=${prefix}/share/info' '--sysconfdir=/etc' '--localstatedir=/var' '--disable-option-checking' '--disable-silent-rules' '--libdir=${prefix}/lib/x86_64-linux-gnu' '--runstatedir=/run' '--disable-maintainer-mode' '--disable-dependency-tracking' 'BUILDCXXFLAGS=-g -O2 -ffile-prefix-map=/home/builder/ubuntu22/build/squid/squid-6.8=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wno-error=deprecated-declarations -Wdate-time -D_FORTIFY_SOURCE=2 -Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -Wl,-z,now ' 'BUILDCXX=g++' '--with-build-environment=default' '--enable-build-info=Ubuntu linux' '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' '--libexecdir=/usr/lib/squid' '--mandir=/usr/share/man' '--enable-inline' '--disable-arch-native' '--enable-async-io=8' '--enable-storeio=ufs,aufs,diskd,rock' '--enable-removal-policies=lru,heap' '--enable-delay-pools' '--enable-cache-digests' '--enable-icap-client' '--enable-follow-x-forwarded-for' '--enable-auth-basic=DB,fake,getpwnam,LDAP,NCSA,PAM,POP3,RADIUS,SASL,SMB' '--enable-auth-digest=file,LDAP' '--enable-auth-negotiate=kerberos,wrapper' '--enable-auth-ntlm=fake,SMB_LM' '--enable-external-acl-helpers=file_userip,kerberos_ldap_group,LDAP_group,session,SQL_session,time_quota,unix_group,wbinfo_group' '--enable-security-cert-validators=fake' '--enable-storeid-rewrite-helpers=file' '--enable-url-rewrite-helpers=fake' '--enable-eui' '--enable-esi' '--enable-icmp' '--enable-zph-qos' '--enable-ecap' '--disable-translation' '--with-swapdir=/var/spool/squid' '--with-logdir=/var/log/squid' '--with-pidfile=/run/squid.pid' '--with-filedescriptors=65536' '--with-large-files' '--with-default-user=proxy' '--enable-linux-netfilter' '--with-systemd' '--with-gnutls' 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -ffile-prefix-map=/home/builder/ubuntu22/build/squid/squid-6.8=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wno-error=deprecated-declarations' 'LDFLAGS=-Wl,-Bsymbolic-functions -flto=auto -ffat-lto-objects -flto=auto -Wl,-z,relro -Wl,-z,now ' 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 -ffile-prefix-map=/home/builder/ubuntu22/build/squid/squid-6.8=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wno-error=deprecated-declarations'

The GnuTLS exception is thrown at my parent proxy. Unfortunately, I cannot make any changes here. So yes, I trust my parent proxy, but not using a tunnel between child and parent does not seem to work and results in the TLS exception on the parent proxy.

I have not find a way to tell my child proxy to always setup a tunnel through the parent proxy, when the target server talks HTTPS. Do you know, how to achieve that? It would be a promising approach.

Thank you very much help and your patience, Alex.

Regards,
Christoph

>-----Ursprüngliche Nachricht-----
>Von: squid-users <squid-users-bounces at lists.squid-cache.org> Im Auftrag von Alex Rousskov
>Gesendet: Donnerstag, 11. Juli 2024 18:15
>An: squid-users at lists.squid-cache.org
>Betreff: Re: [squid-users] Rewriting HTTP to HTTPS for generic package proxy
>
>On 2024-07-10 16:57, Fiehe, Christoph wrote:
>
>> I am just trying to find something that helps to narrow down the
>> problem. What I want to achieve is, that a client can use HTTP in the
>> LAN, so that Squid can cache distribution packages without making use
>> of SSL intercepting when repos are only accessible via HTTPS.
>
>OK.
>
>
>> In that case the secure connection must start at the proxy and end on
>> the target server with or without any upstream proxies in betweem.
>
>It depends on whether you trust the parent proxy:
>
>If you trust the parent proxy, then you can use two secure connections:
>
>1.1. child - parent (TLS; no CONNECT)
>1.2. parent - origin (TLS; no CONNECT)
>
>If you do not trust the parent proxy, then, yes, you will need a tunnel:
>
>2.1. child - parent (CONNECT)
>2.2. child - origin (TLS inside the CONNECT tunnel)
>
>N.B. CONNECT request in 2.1 may be plain text (common) or encrypted
>(rare); I am ignoring the difference between those two subcases for now.
>
>
>> We have the following setup:
>>
>> client -> downstream proxy -> upstream proxy -> https://download.docker.com
>>
>> Now let us assume the client wants to retrieve the following resource
>http://download.docker.com/linux/ubuntu/dists/jammy/InRelease from the upstream proxy.
>>
>> The client initiates a HTTP GET request and sends it to the downstream proxy. Now, the
>URL gets rewritten.
>
>OK.
>
>
>> It indicates to use a HTTPS connection instead in order to talk to the target server, in
>our case the result is https://download.docker.com/linux/ubuntu/dists/jammy/InRelease.
>
>Yes, but HTTPS scheme does not imply that the child Squid has to use
>CONNECT. There are two possible scenarios detailed above. I do not know
>which of them applies to your use case.
>
>
>> Now comes the critical point: From my understanding – it may be
>> wrongof course - the downstream server now has to send a CONNECT
>> request to the upstream server
>
>Yes, provided the child (downstream) proxy does not trust that parent
>(upstream) proxy. That is scenario 2. Scenario 1 is different.
>
>
>> to advise him to establish a secure connection to the target server.
>
>No, the CONNECT tunnel itself is just a pair of TCP connections. The
>parent proxy "secures" nothing but basic TCP connectivity. It is the
>child proxy that negotiates TLS (over/inside that tunnel) with the
>origin server.
>
>
>> After creation, the downstream proxy can retrieve the resource and
>> send it back to the client via plain HTTP.
>
>Yes.
>
>
>
>> I suppose, that the GnuTLS occurs because of a missing SSL handshake
>> between downstream proxy and download.docker.com.
>
>At this time, I can only say that a TLS negotiation error occurs (while
>child Squid is using the encryption library it probably should not be
>using for this). It is not yet clear to me whether child Squid is
>negotiating with the wrong hop or something goes wrong during
>negotiation with the right hop.
>
>As the next steps, I recommend switching to OpenSSL and, if that alone
>does not help, sharing new errors and determining whether you want to
>use scenario 1 (no CONNECT), scenario 2 (CONNECT), or either (whichever
>works): Do you trust the parent Squid?
>
>
>HTH,
>
>Alex.
>
>
>>> -----Ursprüngliche Nachricht-----
>>> Von: Alex Rousskov <rousskov at measurement-factory.com>
>>> Gesendet: Mittwoch, 10. Juli 2024 22:15
>>> An: squid-users at lists.squid-cache.org
>>> Cc: Fiehe, Christoph <c.fiehe at eurodata.de>
>>> Betreff: AW: [squid-users] Rewriting HTTP to HTTPS for generic package proxy
>>>
>>> On 2024-07-10 15:31, Fiehe, Christoph wrote:
>>>> The problem is that the proxy just forwards the client GET request to the upstream
>proxy
>>>
>>> Why does sending a GET request to the upstream proxy represent a problem
>>> in your use case? I cannot find anything in your prior messages on this
>>> thread that would preclude sending a GET request to the upstream proxy.
>>>
>>>
>>>> but in that case a CONNECT is required.
>>>
>>> Why?
>>>
>>> Please do not interpret my response as implying that this "must send
>>> CONNECT" requirement is wrong (or correct). At this point, I am just
>>> trying to understand what problem(s) you are trying to solve beyond the
>>> one you have originally described.
>>>
>>>
>>> Thank you,
>>>
>>> Alex.
>>
>> _______________________________________________
>> squid-users mailing list
>> squid-users at lists.squid-cache.org
>> https://lists.squid-cache.org/listinfo/squid-users
>
>_______________________________________________
>squid-users mailing list
>squid-users at lists.squid-cache.org
>https://lists.squid-cache.org/listinfo/squid-users