[squid-users] Squid fails to pass on HEAD requests to parent

Tue Sep 15 15:18:03 UTC 2015

In our network we are behind a proxy that I don't have access to. In order
to speed up deployments and development I am trying to set up a caching
squid proxy for yum and maven repositories.
Naturally, this proxy needs to be configured to use our company's global
proxy as parent.

I have successfully set it up to the point where it works when e.g.
downloading files using wget. However when using it with an actual maven
build, the build hangs when trying to download pom or jar files.

After having increased the log level I found out that my squid does not use
the parent proxy in such cases, and tries to connect to the internet which
is not possible since we can only connect through the global proxy.
A closer look at the logs revealed that maven issued HEAD instead of GET
requests in my case. I could hence reproduce the problem without maven
using this command line:

curl -I
> http://repo.springsource.org/snapshot/org/springframework/boot/spring-boot-starter-parent/1.2.2.RELEASE/spring-boot-starter-parent-1.2.2.RELEASE.pom

In this example we would normally get a 404 (maven tries out a configured
list of servers to find a particular resource), however the same problem
applies with existing resources.

To me it seems like my squid does not understand it needs to use the global
proxy for HEAD requests as well as for GET. But I could not find any
reference to this particular problem anywhere in the web.

I've appended all information that seems relevant below. Now I would really
like to know: what am I doing wrong?

Cheers,

Martin

*Appendix: system information, log messages and configuration.*

I am using squid 3.1.23 on an Oracle Linux 6.7 system (an RHEL6 variant). I
have reproduced the same problem on an Oracle Linux 7.1 system with squid
3.5.3 with basically the same configuration.

Here's a snippet of what I find in the log after such an unsuccessful
request:

2015/09/15 12:29:06.364| peerSelectFoo: 'HEAD repo.springsource.org'
> 2015/09/15 12:29:06.364| peerSelectFoo: direct = DIRECT_MAYBE
> 2015/09/15 12:29:06.364| peerSelectIcpPing:
> http://repo.springsource.org/snapshot/org/springframework/boot/spring-boot-starter-parent/1.2.2.RELEASE/spring-boot-starter-parent-1.2.2.RELEASE.pom
> 2015/09/15 12:29:06.364| peerAddFwdServer: adding DIRECT DIRECT
> 2015/09/15 12:29:06.364| peerSelectCallback:
> http://repo.springsource.org/snapshot/org/springframework/boot/spring-boot-starter-parent/1.2.2.RELEASE/spring-boot-starter-parent-1.2.2.RELEASE.pom
> 2015/09/15 12:29:06.364| cbdataReferenceValid: 0x7fa2dbf5f3c8
> 2015/09/15 12:29:06.364| cbdataUnlock: 0x7fa2dbf5f3c8=1
> 2015/09/15 12:29:06.364| fwdStartComplete:
> http://repo.springsource.org/snapshot/org/springframework/boot/spring-boot-starter-parent/1.2.2.RELEASE/spring-boot-starter-parent-1.2.2.RELEASE.pom
> 2015/09/15 12:29:06.364| fwdConnectStart:
> http://repo.springsource.org/snapshot/org/springframework/boot/spring-boot-starter-parent/1.2.2.RELEASE/spring-boot-starter-parent-1.2.2.RELEASE.pom
> 2015/09/15 12:29:06.364| PconnPool::key(repo.springsource.org,80,(no
> domain),[::]is {repo.springsource.org:80}

In contrast, when I issue the above command without '-I', I get a different
log output:

2015/09/15 12:32:54.348| peerSelectFoo: 'GET repo.springsource.org'
> 2015/09/15 12:32:54.348| peerSelectFoo: direct = DIRECT_MAYBE
> 2015/09/15 12:32:54.348| peerDigestLookup: peer proxy.local.lan
> 2015/09/15 12:32:54.348| peerDigestLookup: gone!
> 2015/09/15 12:32:54.348| neighborsDigestSelect: choices: 0 (0)
> 2015/09/15 12:32:54.348| peerNoteDigestLookup: peer <none>, lookup:
> LOOKUP_NONE
> 2015/09/15 12:32:54.348| peerSelectIcpPing:
> http://repo.springsource.org/snapshot/org/springframework/boot/spring-boot-starter-parent/1.2.2.RELEASE/spring-boot-starter-parent-1.2.2.RELEASE.pom
> 2015/09/15 12:32:54.348| neighborsCount: 0
> 2015/09/15 12:32:54.348| peerSelectIcpPing: counted 0 neighbors
> 2015/09/15 12:32:54.348| peerGetSomeParent: GET repo.springsource.org
> 2015/09/15 12:32:54.348| neighbors.cc(339) getRoundRobinParent: returning
> NULL
> 2015/09/15 12:32:54.348| getWeightedRoundRobinParent: returning NULL
> 2015/09/15 12:32:54.348| neighborUp: UP (no-query): proxy.local.lan (
> 172.16.8.250:3130)
> 2015/09/15 12:32:54.348| neighborUp: UP (no-query): proxy.local.lan (
> 172.16.8.250:3130)
> 2015/09/15 12:32:54.348| getFirstUpParent: returning proxy.local.lan
> 2015/09/15 12:32:54.348| peerSelect: FIRST_UP_PARENT/proxy.local.lan
> 2015/09/15 12:32:54.348| peerAddFwdServer: adding
> proxy.local.lan FIRST_UP_PARENT
> 2015/09/15 12:32:54.348| cbdataLock: 0x7fa2dbca0d58=1
> 2015/09/15 12:32:54.348| peerAddFwdServer: adding DIRECT DIRECT
> 2015/09/15 12:32:54.348| peerSelectCallback:
> http://repo.springsource.org/snapshot/org/springframework/boot/spring-boot-starter-parent/1.2.2.RELEASE/spring-boot-starter-parent-1.2.2.RELEASE.pom
> 2015/09/15 12:32:54.348| cbdataReferenceValid: 0x7fa2dbf5f3c8
> 2015/09/15 12:32:54.348| cbdataUnlock: 0x7fa2dbf5f3c8=1
> 2015/09/15 12:32:54.348| fwdStartComplete:
> http://repo.springsource.org/snapshot/org/springframework/boot/spring-boot-starter-parent/1.2.2.RELEASE/spring-boot-starter-parent-1.2.2.RELEASE.pom

As we see, in the second example the parent proxy is used, while in the
first it is not (and hence trying to connect repo.springsource.org fails).

Here is what I changed to the default configuration:

#acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
> #acl localnet src 192.168.0.0/16        # RFC1918 possible internal
> network
> [...]

cache_dir ufs /var/cache/squid 1000 16 256
>
[...]

cache_mem 64 MB
> cache_log /var/log/squid/cache.log
> cache_store_log /var/log/squid/store.log
> cache_effective_user squid
> cache_effective_group squid
> emulate_httpd_log on
> debug_options ALL,10
> refresh_pattern ^ftp:           1440    20%     10080
> refresh_pattern ^gopher:        1440    0%      1440
> refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
> refresh_pattern .               0       20%     4320
> cache_peer proxy.dermalog.hh parent 3128 3130 no-query no-digest
> no-netdb-exchange
> prefer_direct off

-- 
---------- MDietze at gmail.com --/-- martin at the-little-red-haired-girl.org
----
------------- / http://herbert.the-little-red-haired-girl.org /
-------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20150915/ea915047/attachment-0001.html>