[squid-users] Ubuntu 20.04 "apt update" issues behind a VPN and Squid proxy

Amos Jeffries squid3 at treenet.co.nz
Thu Jul 8 02:17:13 UTC 2021 


On 8/07/21 11:44 am, David Mills wrote:
> Hi Eliezer,
> 
> We have:
> 
> /etc/apt/apt.conf:
> 
>     Acquire::http::proxy
>     "http://vpn-proxy-d68aca8a8f7f81d6.elb.ap-southeast-2.amazonaws.com:3128/
>     <http://vpn-proxy-d68aca8a8f7f81d6.elb.ap-southeast-2.amazonaws.com:3128/>";
>     Acquire::https::proxy
>     "http://vpn-proxy-d68aca8a8f7f81d6.elb.ap-southeast-2.amazonaws.com:3128/
>     <http://vpn-proxy-d68aca8a8f7f81d6.elb.ap-southeast-2.amazonaws.com:3128/>";
> 
> 
> /etc/apt/sources.list (comment lines removed for brevity)
> 
>     deb https://mirror.aarnet.edu.au/ubuntu/
>     <https://mirror.aarnet.edu.au/ubuntu/> focal main restricted
>     deb https://mirror.aarnet.edu.au/ubuntu/
>     <https://mirror.aarnet.edu.au/ubuntu/> focal-updates main restricted
>     deb https://mirror.aarnet.edu.au/ubuntu/
>     <https://mirror.aarnet.edu.au/ubuntu/> focal-updates universe
>     deb https://mirror.aarnet.edu.au/ubuntu/
>     <https://mirror.aarnet.edu.au/ubuntu/> focal multiverse
>     deb https://mirror.aarnet.edu.au/ubuntu/
>     <https://mirror.aarnet.edu.au/ubuntu/> focal-updates multiverse
>     deb https://mirror.aarnet.edu.au/ubuntu/
>     <https://mirror.aarnet.edu.au/ubuntu/> focal-backports main
>     restricted universe multiverse
>     deb https://mirror.aarnet.edu.au/ubuntu
>     <https://mirror.aarnet.edu.au/ubuntu> focal-security main restricted
>     deb https://mirror.aarnet.edu.au/ubuntu
>     <https://mirror.aarnet.edu.au/ubuntu> focal-security universe
>     deb https://mirror.aarnet.edu.au/ubuntu
>     <https://mirror.aarnet.edu.au/ubuntu> focal-security multiverse
> 
> 
> squid.conf
> 
...
>     #
>     # INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
>     #
> 
>     # Redirect HTTP to HTTPS
>     acl port_80 port 80
>     acl gstatic dstdomain www.gstatic.com <http://www.gstatic.com>
>     http_access deny port_80 gstatic
>     deny_info 301:https://%H%R gstatic
> 
>     acl avpc dstdomain crop-assessment.acusensus-vpc
>     http_access deny port_80 avpc
>     deny_info 302:<company url> avpc
> 
> 
>     # Deny HTTP
>     http_access deny port_80
> 
>     # Whitelist of allowed sites
>     acl allowed_http_sites dstdomain "/etc/squid/squid.allowed.sites.txt"
>     http_access allow allowed_http_sites vpc_cidr
> 

Is the "mirror.aarnet.edu.au" or a wildcard matching it listed in file 
squid.allowed.sites.txt ?

(I assume so, but checking in case it is that simple).


>     # And finally deny all other access to this proxy
>     http_access deny all
> 
>     # Squid normally listens to port 3128
>     http_port 3128 ssl-bump cert=/etc/squid/cert.pem
>     acl allowed_https_sites ssl::server_name
>     "/etc/squid/squid.allowed.sites.txt"
>     acl step1 at_step SslBump1
>     acl step2 at_step SslBump2
>     acl step3 at_step SslBump3
>     ssl_bump peek step1 all
>     ssl_bump peek step2 allowed_https_sites
>     ssl_bump splice step3 allowed_https_sites
>     ssl_bump terminate step2 all
> 
>     # Uncomment and adjust the following to add a disk cache directory.
>     #cache_dir ufs /var/spool/squid 100 16 256
> 
>     # Leave coredumps in the first cache dir
>     coredump_dir /var/spool/squid
>     #
>     # Add any of your own refresh_pattern entries above these.
>     #
>     refresh_pattern ^ftp: 1440 20% 10080
>     refresh_pattern ^gopher: 1440 0% 1440
>     refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
>     refresh_pattern . 0 20% 4320
> 
> 
> 
> Squid 3.5 is running on an EC2 instance running Amazon Linux 2. I'll 
> answer the questions you asked Ben for extra info.
> ip address:
> 
>     1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
>     group default qlen 1000
>          link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
>          inet 127.0.0.1/8 <http://127.0.0.1/8> scope host lo
>             valid_lft forever preferred_lft forever
>          inet6 ::1/128 scope host
>             valid_lft forever preferred_lft forever
>     2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state
>     UP group default qlen 1000
>          link/ether 02:1b:15:b8:9a:06 brd ff:ff:ff:ff:ff:ff
>          inet 10.0.12.111/24 <http://10.0.12.111/24> brd 10.0.12.255
>     scope global dynamic eth0
>             valid_lft 2393sec preferred_lft 2393sec
>          inet6 fe80::1b:15ff:feb8:9a06/64 scope link
>             valid_lft forever preferred_lft forever
> 
> 
> ip rule
> 
>     0: from all lookup local
>     32766: from all lookup main
>     32767: from all lookup default 
> 
> 
> ip route show
> 
>     default via 10.0.12.1 dev eth0
>     10.0.12.0/24 <http://10.0.12.0/24> dev eth0 proto kernel scope link
>     src 10.0.12.111
>     169.254.169.254 dev eth0
> 
> 


The traffic from Squid to the AArnet server is apparently using IPv6. Is 
that routing setup properly too?


...

>     From: squid-users On Behalf Of David Mills
>     Sent: Wednesday, July 7, 2021 2:26 AM
...
>     We have tried upgrading one to 20.04. Same setup. From the command
>     line curl or wget can happily download an Ubuntu package from the
>     Ubuntu Mirror site we use. But "apt update" gets lots of "IGN:"
>     timeouts and errors.
> 
>     The package we test curl with is
>     https://mirror.aarnet.edu.au/ubuntu/pool/main/c/curl/curl_7.68.0-1ubuntu2.5_amd64.deb
>     <https://mirror.aarnet.edu.au/ubuntu/pool/main/c/curl/curl_7.68.0-1ubuntu2.5_amd64.deb>
> 
>     The Squid log shows a line the doesn't occur for the successful
>     18.04 "apt updates":
>     1625190959.233     81 10.0.11.191 TAG_NONE/200 0 CONNECT
>     http://mirror.aarnet.edu.au:443 <http://mirror.aarnet.edu.au:443> -
>     HIER_DIRECT/2001:388:30bc:cafe::beef -
> 

With Ubuntu 20.04 you should have received Squid-4 (v4.10 or later). 
Which logs a few things differently from Squid-3.5, particularly for 
SSL-Bump activity and client connections that lack HTTP messages.

The above log line shows SSL-Bump activity. At least step2 was reached, 
possibly also step3. Looking at this a little closer to see if it 
completes fine or has unseen issues would be my next point of approach.

To debug what is happening with SSL-Bump use "debug_options ALL1, 11,2 
83,5" and check the resulting cache.log.



>     The full output of an attempt to update is:
>     Ign:1 https://mirror.aarnet.edu.au/ubuntu
>     <https://mirror.aarnet.edu.au/ubuntu> focal InRelease
>     Ign:2 https://mirror.aarnet.edu.au/ubuntu
>     <https://mirror.aarnet.edu.au/ubuntu> focal-updates InRelease
>     Ign:3 https://mirror.aarnet.edu.au/ubuntu
>     <https://mirror.aarnet.edu.au/ubuntu> focal-backports InRelease
>     Ign:4 https://mirror.aarnet.edu.au/ubuntu
>     <https://mirror.aarnet.edu.au/ubuntu> focal-security InRelease


These "Ign" are fine. They just mean that apt has determined those files 
it has cached are up-to-date and do not need to be re-fetched right now.

The below "Err" are the problem:

>     Err:5 https://mirror.aarnet.edu.au/ubuntu
>     <https://mirror.aarnet.edu.au/ubuntu> focal Release
>        Could not wait for server fd - select (11: Resource temporarily
>     unavailable) [IP: 10.0.11.82 3128]...
> 
>     While running, the line
>     0% [Connecting to HTTP proxy
>     (http://vpn-proxy-d68aca8a8f7f81d6.elb.ap-southeast-2.amazonaws.com:3128
>     <http://vpn-proxy-d68aca8a8f7f81d6.elb.ap-southeast-2.amazonaws.com:3128>)]
>     appears often and hang for a while.
> 
>     I've tried upping the squid logging and allowing all, but they
>     didn't offer any additional information about the issue.
> 

Your squid.conf looks fine, assuming the same http_access rules were 
used in your working version.


I suspect the issue is related to one or more of:

  * IPv6 routing

  * ICMP config issues (maybe outside your network)

  * SSL-Bump issues processing the client or server handshake traffic
    typically seen with OpenSSL library version or config mismatches 
between Squid, client and server.

  * network timeouts affecting Squid


HTH
Amos

More information about the squid-users mailing list