[squid-users] allow request to cloudfront after 302 redirection.

[Alex Rousskov]

On 4/21/21 12:48 PM, Miroslaw Malinowski wrote:
> Is it possible to create a whitelist that allows cloudfront 302
> redirections, e.g. gitlab is using cloudfront as CDN and when we
> whitelist package.gitlab.com the URL is redirected (302) to
> https://d20rj4el6vkp4c.cloudfront.net/7/11/ubuntu/package_files/35938.deb?t=1619023239_a63698472b6bebeaee980e7c030632d97a29c15d


Yes, it is possible to allow future requests to Location-listed URLs,
but since we are talking about two (or more) independent HTTP
transactions, on two (or more) TCP connections, you will need to store
the allowed Location values (at least) somewhere, maintain that storage
(e.g., remove stale entries), and (optionally) determine whether the
request for an allowed cloudfront URL came from the same user agent as
the gitlab request that was redirected to that URL.

Storing, maintenance, and checking of allowed Locations/etc. can be done
using external ACLs and/or eCAP/ICAP adaptation services. It cannot be
reliably done using built-in ACLs alone AFAICT.


HTH,

Alex.


> I could whitelist a whole .cloudfront.net <http://cloudfront.net> domain
> or url_regex, but what I would like to achieve, I don't know if
> possible, is a chain of events like:
> If packages.gitlab.com <http://packages.gitlab.com> return 302 Location
> .cloudfront, then allow
> https://d20rj4el6vkp4c.cloudfront.net/7/11/ubuntu/package_files/35938.deb?t=1619023239_a63698472b6bebeaee980e7c030632d97a29c
> <https://d20rj4el6vkp4c.cloudfront.net/7/11/ubuntu/package_files/35938.deb?t=1619023239_a63698472b6bebeaee980e7c030632d97a29c>
> request.
> I've been playing around with http_reply_access and rep_headers, but I
> can only go as far as allow replay of the first request to
> package.gitlab.com <http://package.gitlab.com>, but then a GET to
> cloudfront is blocked anyway as it's not on our whitelist.
> e.g.
> 1619022938.916   423 172.16.230.237 NONE/200 0 CONNECT 54.153.54.194:443
> <http://54.153.54.194:443> - ORIGINAL_DST/54.153.54.194
> <http://54.153.54.194> -
> 1619022939.074   153 172.16.230.237 TCP_MISS/302 758 GET
> https://packages.gitlab.com/gitlab/gitlab-ee/packages/ubuntu/bionic/gitlab-ee_11.0.1-ee.0_amd64.deb/download.deb
> <https://packages.gitlab.com/gitlab/gitlab-ee/packages/ubuntu/bionic/gitlab-ee_11.0.1-ee.0_amd64.deb/download.deb>
> - ORIGINAL_DST/54.153.54.194 <http://54.153.54.194> text/html
> 1619022939.108    20 172.16.230.237 NONE/200 0 CONNECT 52.84.90.34:443
> <http://52.84.90.34:443> - ORIGINAL_DST/52.84.90.34 <http://52.84.90.34> -
> 1619022939.114     2 172.16.230.237 TCP_DENIED/403 19053 GET
> https://d20rj4el6vkp4c.cloudfront.net/7/11/ubuntu/package_files/35938.deb <https://d20rj4el6vkp4c.cloudfront.net/7/11/ubuntu/package_files/35938.deb>?
> - HIER_NONE/- text/html