[squid-users] URL list from a URL

[Eliezer Croitoru]

And if you want the full list of mirrors being updated instead of using curl you can use the next script to parse the full mirror list page:
http://mirror-status.centos.org/

Using the script:
https://gist.github.com/elico/dc1af72344231d6d49af4eacecfae8df#file-update-mirrors-list-sh

Let me know if it fills the gaps.

Eliezer

----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer at ngtech.co.il



-----Original Message-----
From: squid-users [mailto:squid-users-bounces at lists.squid-cache.org] On Behalf Of Eliezer Croitoru
Sent: Friday, March 24, 2017 4:33 PM
To: 'Jason B. Nance' <jason at tresgeek.net>; squid-users at lists.squid-cache.org
Subject: Re: [squid-users] URL list from a URL

OK so I wrote these helpers that can help to allow the updates despite the changes in urls on gist:
https://gist.github.com/elico/dc1af72344231d6d49af4eacecfae8df

The squid.conf should be something like:
external_acl_type centos_mirror_check ipv4 concurrency=200 ttl=15 %URI %SRC %METHOD /opt/bin/centos-extacl.rb /etc/squid/centosrepos.txt
acl centos_mirror_check_acl external centos_mirror_check
htttp_access allow !CONNECT centos_mirror_check_acl


Let me know if this is not enough or there is a need for some tweaks.

Eliezer

----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer at ngtech.co.il



-----Original Message-----
From: squid-users [mailto:squid-users-bounces at lists.squid-cache.org] On Behalf Of Jason B. Nance
Sent: Tuesday, March 21, 2017 5:53 PM
To: squid-users at lists.squid-cache.org
Subject: [squid-users] URL list from a URL

Hello,

I'm using Squid 3.5.20 and wonder if it is possible to define an ACL which retrieves the list of URLs from another URL (similar to pointing to a file).  In this specific use case it is to allow a Foreman server to sync Yum content from the CentOS mirrors.  I tell Foreman to use the following URL:

http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=updates

Which returns a list of URLs, such as:

http://repo1.dal.innoscale.net/centos/7.3.1611/updates/x86_64/
http://linux.mirrors.es.net/centos/7.3.1611/updates/x86_64/
http://reflector.westga.edu/repos/CentOS/7.3.1611/updates/x86_64/
http://mirror.jax.hugeserver.com/centos/7.3.1611/updates/x86_64/
http://ftp.linux.ncsu.edu/pub/CentOS/7.3.1611/updates/x86_64/
http://mirror.nexcess.net/CentOS/7.3.1611/updates/x86_64/
http://mirror.web-ster.com/centos/7.3.1611/updates/x86_64/
http://centos.host-engine.com/7.3.1611/updates/x86_64/
http://mirror.raystedman.net/centos/7.3.1611/updates/x86_64/
http://mirror.linux.duke.edu/pub/centos/7.3.1611/updates/x86_64/

Foreman then starts a new HTTP connection (not a redirect) to attempt to connect to those in turn until it works.

So I would like to configure Squid to allow the Foreman server access to any of those URLs (the list changes somewhat often).

I started to go down the external_acl_type but am wondering if I'm missing something obvious.

Regards,

j
_______________________________________________
squid-users mailing list
squid-users at lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
squid-users at lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users