[squid-users] Lots of "Vary object loop!"
Sebastián Goicochea
sebag at vianetcon.com.ar
Thu Sep 3 15:20:30 UTC 2015
Amos, I spent a couple of days doing some test with the info you gave me:
Retested emptying the cache several times, disabled the rewriter,
different config files .. all I could think of
Downloaded fresh 3.5.8 tar.gz (just in case it was some 3.5.4 thing) and
compiled it using this configure options:
Squid Cache: Version 3.5.8
Service Name: squid
configure options: '--prefix=/usr/local' '--datadir=/usr/local/share'
'--bindir=/usr/local/sbin' '--libexecdir=/usr/local/lib/squid'
'--localstatedir=/var' '--sysconfdir=/etc/squid3' '--enable-delay-pools'
'--enable-ssl' '--enable-ssl-crtd' '--enable-linux-netfilter'
'--enable-eui' '--enable-snmp' '--enable-gnuregex'
'--enable-ltdl-convenience' '--enable-removal-policies=lru heap'
'--enable-http-violations' '--with-openssl'
'--with-filedescriptors=24321' '--enable-poll' '--enable-epoll'
'--enable-storeio=ufs,aufs,diskd,rock' '--disable-ipv6'
And the problem appeared again, I am suspicious that the problem is in
the configuration, I even removed all my refresh patterns, but:
2015/09/02 15:03:42 kid1| varyEvaluateMatch: Oops. Not a Vary match on
second attempt, 'http://assets.pinterest.com/js/pinit.js'
'accept-encoding="gzip,%20deflate"'
2015/09/02 15:03:42 kid1| clientProcessHit: Vary object loop!
2015/09/02 15:03:43 kid1| varyEvaluateMatch: Oops. Not a Vary match on
second attempt, 'http://static.cmptch.com/v/lib/str.html'
'accept-encoding="gzip,%20deflate,%20sdch"'
2015/09/02 15:03:43 kid1| clientProcessHit: Vary object loop!
2015/09/02 15:03:43 kid1| varyEvaluateMatch: Oops. Not a Vary match on
second attempt,
'http://pstatic.bestpriceninja.com/nwp/v0_0_773/release/Shared/Extra/IFrameStoreReciever.js'
'accept-encoding="gzip,%20deflate,%20sdch"'
2015/09/02 15:03:43 kid1| clientProcessHit: Vary object loop!
2015/09/02 15:03:59 kid1| varyEvaluateMatch: Oops. Not a Vary match on
second attempt,
'http://static.xvideos.com/v2/css/xv-video-styles.css?v=7'
'accept-encoding="gzip,deflate"'
2015/09/02 15:03:59 kid1| clientProcessHit: Vary object loop!
2015/09/02 15:03:59 kid1| varyEvaluateMatch: Oops. Not a Vary match on
second attempt, 'http://s7.addthis.com/js/250/addthis_widget.js'
'accept-encoding="gzip,deflate"'
2015/09/02 15:03:59 kid1| clientProcessHit: Vary object loop!
Later on I tested it with this short config file and the problem persisted:
http_access allow localhost manager
http_access deny manager
acl purge method PURGE
http_access allow purge localhost
http_access deny purge
acl all src all
acl localhost src 127.0.0.1/32
acl localnet src 127.0.0.0/8
acl Safe_ports port 80
acl snmppublic snmp_community public
http_access deny !Safe_ports
http_access allow all
dns_v4_first on
cache_mem 1024 MB
maximum_object_size_in_memory 64 KB
memory_cache_mode always
maximum_object_size 150000 KB
minimum_object_size 100 bytes
collapsed_forwarding on
logfile_rotate 5
mime_table /etc/squid3/mime.conf
debug_options ALL,1
store_id_access deny all
store_id_bypass on
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern ^http:\/\/movies\.apple\.com 86400 20%
86400 override-expire override-lastmod ignore-no-cache ignore-private
ignore-reload
refresh_pattern -i \.flv$ 10080 90% 999999
ignore-no-cache override-expire ignore-private
refresh_pattern -i \.mov$ 10080 90% 999999
ignore-no-cache override-expire ignore-private
refresh_pattern windowsupdate.com/.*\.(cab|exe) 4320 100% 43200
reload-into-ims
refresh_pattern download.microsoft.com/.*\.(cab|exe) 4320 100% 43200
reload-into-ims
refresh_pattern -i
\.(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|pdf|tiff)$ 10080 90%
43200 override-expire ignore-no-cache ignore-private
refresh_pattern -i (/cgi-bin/) 0 0% 0
refresh_pattern . 0 20% 4320
quick_abort_min 0 KB
quick_abort_max 0 KB
quick_abort_pct 100
range_offset_limit 0
negative_ttl 1 minute
negative_dns_ttl 1 minute
read_ahead_gap 128 KB
request_header_max_size 100 KB
reply_header_max_size 100 KB
via off
acl apache rep_header Server ^Apache
half_closed_clients off
cache_mgr webmaster
cache_effective_user squid
cache_effective_group squid
httpd_suppress_version_string on
snmp_access allow snmppublic localhost
snmp_access deny all
snmp_incoming_address 127.0.0.1
error_directory /etc/squid3/errors/English
max_filedescriptors 65535
ipcache_size 1024
forwarded_for off
log_icp_queries off
icp_access allow localnet
icp_access deny all
htcp_access allow localnet
htcp_access deny all
digest_rebuild_period 15 minutes
digest_rewrite_period 15 minutes
strip_query_terms off
max_open_disk_fds 150
cache_replacement_policy heap LFUDA
memory_pools off
http_port 9001
http_port 901 tproxy
if ${process_number} = 1
access_log stdio:/var/log/squid/1/access.log squid
cache_log /var/log/squid/1/cache.log
cache_store_log none
cache_swap_state /var/log/squid/1/%s.swap.state
else
access_log none
cache_log /dev/null
endif
pid_filename /var/run/squid1.pid
visible_hostname localhost
snmp_port 1611
icp_port 3131
htcp_port 4828
cachemgr_passwd admin thisisnotmyrealpassword
memory_cache_shared off
cache_dir rock /cache1/rock1 256 min-size=100 max-size=3000
cache_dir rock /cache1/rock2 2000 min-size=3000 max-size=20000
cache_dir diskd /cache1/diskd2 60000 16 256 min-size=20000 max-size=200000
cache_dir diskd /cache2/2 100000 16 256 min-size=200000 max-size=1048576
cache_dir diskd /cache2/1 680000 16 256 min-size=1048576
Any ideas what could be wrong?
Thanks,
Sebastian
El 26/08/15 a las 17:15, Amos Jeffries escribió:
> On 27/08/2015 7:53 a.m., Sebastián Goicochea wrote:
>> After I sent you my previous email, I continued investigating the
>> subject .. I made a change in the source code as follows:
>>
>> File: /src/http.cc
>>
>> HttpStateData::haveParsedReplyHeaders()
>> {
>> .
>> .
>> ##### THIS IS NEW STUFF ###########
>> if (rep->header.has(HDR_VARY)) {
>> rep->header.delById(HDR_VARY);
>> debugs(11,3, "Vary detected. Hack Cleaning it up");
>> }
>> ##### END OF NEW STUFF ###########
>>
>> #if X_ACCELERATOR_VARY
>> if (rep->header.has(HDR_X_ACCELERATOR_VARY)) {
>> rep->header.delById(HDR_X_ACCELERATOR_VARY);
>> debugs(11,3, "HDR_X_ACCELERATOR_VARY Vary detected. Hack Cleaning it
>> up");
>> }
>> #endif
>> .
>> .
>>
>>
>> Deleting Vary from the header at this point gives me hits in every
>> object I test (that previously didn't hit) .. web browser never receives
>> the Vary in the response header.
>> Now I read your answer and you say that this is a critical validity
>> check and that worries me. Taking away the vary altogether at this point
>> could lead to the problems that you described? If that is the case .. I
>> have to investigate other alternatives.
>>
> I'll have to look into that function when I'm back at the code later to
> confirm this. But IIRC that function is acting directly on a freshly
> received reply message. You are not removing the validity check, you are
> removing Squids ability to see that it is a Vary object at all. So it is
> never even cached as one.
>
> The side effect of that is that clients asking for non-gzip can get the
> cached gzip copy, etc. but at least its the same URL. So the security
> risks are gone. But the user experience is not always good either way.
>
> Amos
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20150903/4f1474a6/attachment.html>
More information about the squid-users
mailing list