[squid-users] allow request to cloudfront after 302 redirection.

Mon May 24 09:52:04 UTC 2021

Hey Alex,

Following up this thread I was wondering about an example how to do that with an external_acl helper.
With ICAP I can do that easily to some degree.
With an external_acl helper I am not sure what values to send.
I would guess that the response code and response Location header might be the ones which should be passed to the helper?

What do you think about the next acls, should do the trick?(code to be followed)

acl redirect http_status 301-308
acl gitlab_package dstdomain package.gitlab.com

external_acl_type openlocation children=15 %DST %SRC %<{Location} /usr/local/bin/location-openner.rb
acl location_openner external openlocation
http_access deny gitlab_package redirect location_openner
http_access allow location_openner

Thanks,
----
Eliezer Croitoru
Tech Support
Mobile: +972-5-28704261
Email: ngtech1ltd at gmail.com

-----Original Message-----
From: squid-users <squid-users-bounces at lists.squid-cache.org> On Behalf Of Alex Rousskov
Sent: Wednesday, April 21, 2021 8:49 PM
To: squid-users at lists.squid-cache.org
Subject: Re: [squid-users] allow request to cloudfront after 302 redirection.

On 4/21/21 12:48 PM, Miroslaw Malinowski wrote:
> Is it possible to create a whitelist that allows cloudfront 302
> redirections, e.g. gitlab is using cloudfront as CDN and when we
> whitelist package.gitlab.com the URL is redirected (302) to
> https://d20rj4el6vkp4c.cloudfront.net/7/11/ubuntu/package_files/35938.deb?t=1619023239_a63698472b6bebeaee980e7c030632d97a29c15d

Yes, it is possible to allow future requests to Location-listed URLs,
but since we are talking about two (or more) independent HTTP
transactions, on two (or more) TCP connections, you will need to store
the allowed Location values (at least) somewhere, maintain that storage
(e.g., remove stale entries), and (optionally) determine whether the
request for an allowed cloudfront URL came from the same user agent as
the gitlab request that was redirected to that URL.

Storing, maintenance, and checking of allowed Locations/etc. can be done
using external ACLs and/or eCAP/ICAP adaptation services. It cannot be
reliably done using built-in ACLs alone AFAICT.

HTH,

Alex.

> I could whitelist a whole .cloudfront.net <http://cloudfront.net> domain
> or url_regex, but what I would like to achieve, I don't know if
> possible, is a chain of events like:
> If packages.gitlab.com <http://packages.gitlab.com> return 302 Location
> .cloudfront, then allow
> https://d20rj4el6vkp4c.cloudfront.net/7/11/ubuntu/package_files/35938.deb?t=1619023239_a63698472b6bebeaee980e7c030632d97a29c
> <https://d20rj4el6vkp4c.cloudfront.net/7/11/ubuntu/package_files/35938.deb?t=1619023239_a63698472b6bebeaee980e7c030632d97a29c>
> request.
> I've been playing around with http_reply_access and rep_headers, but I
> can only go as far as allow replay of the first request to
> package.gitlab.com <http://package.gitlab.com>, but then a GET to
> cloudfront is blocked anyway as it's not on our whitelist.
> e.g.
> 1619022938.916   423 172.16.230.237 NONE/200 0 CONNECT 54.153.54.194:443
> <http://54.153.54.194:443> - ORIGINAL_DST/54.153.54.194
> <http://54.153.54.194> -
> 1619022939.074   153 172.16.230.237 TCP_MISS/302 758 GET
> https://packages.gitlab.com/gitlab/gitlab-ee/packages/ubuntu/bionic/gitlab-ee_11.0.1-ee.0_amd64.deb/download.deb
> <https://packages.gitlab.com/gitlab/gitlab-ee/packages/ubuntu/bionic/gitlab-ee_11.0.1-ee.0_amd64.deb/download.deb>
> - ORIGINAL_DST/54.153.54.194 <http://54.153.54.194> text/html
> 1619022939.108    20 172.16.230.237 NONE/200 0 CONNECT 52.84.90.34:443
> <http://52.84.90.34:443> - ORIGINAL_DST/52.84.90.34 <http://52.84.90.34> -
> 1619022939.114     2 172.16.230.237 TCP_DENIED/403 19053 GET
> https://d20rj4el6vkp4c.cloudfront.net/7/11/ubuntu/package_files/35938.deb <https://d20rj4el6vkp4c.cloudfront.net/7/11/ubuntu/package_files/35938.deb>?
> - HIER_NONE/- text/html

_______________________________________________
squid-users mailing list
squid-users at lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users