[squid-users] Lots of "Vary object loop!"

Sebastián Goicochea sebag at vianetcon.com.ar
Thu Sep 3 15:20:30 UTC 2015


Amos, I spent a couple of days doing some test with the info you gave me:

Retested emptying the cache several times, disabled the rewriter, 
different config files .. all I could think of


Downloaded fresh 3.5.8 tar.gz (just in case it was some 3.5.4 thing) and 
compiled it using this configure options:

Squid Cache: Version 3.5.8
Service Name: squid
configure options:  '--prefix=/usr/local' '--datadir=/usr/local/share' 
'--bindir=/usr/local/sbin' '--libexecdir=/usr/local/lib/squid' 
'--localstatedir=/var' '--sysconfdir=/etc/squid3' '--enable-delay-pools' 
'--enable-ssl' '--enable-ssl-crtd' '--enable-linux-netfilter' 
'--enable-eui' '--enable-snmp' '--enable-gnuregex' 
'--enable-ltdl-convenience' '--enable-removal-policies=lru heap' 
'--enable-http-violations' '--with-openssl' 
'--with-filedescriptors=24321' '--enable-poll' '--enable-epoll' 
'--enable-storeio=ufs,aufs,diskd,rock' '--disable-ipv6'



And the problem appeared again, I am suspicious that the problem is in 
the configuration, I even removed all my refresh patterns, but:

2015/09/02 15:03:42 kid1| varyEvaluateMatch: Oops. Not a Vary match on 
second attempt, 'http://assets.pinterest.com/js/pinit.js' 
'accept-encoding="gzip,%20deflate"'
2015/09/02 15:03:42 kid1| clientProcessHit: Vary object loop!
2015/09/02 15:03:43 kid1| varyEvaluateMatch: Oops. Not a Vary match on 
second attempt, 'http://static.cmptch.com/v/lib/str.html' 
'accept-encoding="gzip,%20deflate,%20sdch"'
2015/09/02 15:03:43 kid1| clientProcessHit: Vary object loop!
2015/09/02 15:03:43 kid1| varyEvaluateMatch: Oops. Not a Vary match on 
second attempt, 
'http://pstatic.bestpriceninja.com/nwp/v0_0_773/release/Shared/Extra/IFrameStoreReciever.js' 
'accept-encoding="gzip,%20deflate,%20sdch"'
2015/09/02 15:03:43 kid1| clientProcessHit: Vary object loop!
2015/09/02 15:03:59 kid1| varyEvaluateMatch: Oops. Not a Vary match on 
second attempt, 
'http://static.xvideos.com/v2/css/xv-video-styles.css?v=7' 
'accept-encoding="gzip,deflate"'
2015/09/02 15:03:59 kid1| clientProcessHit: Vary object loop!
2015/09/02 15:03:59 kid1| varyEvaluateMatch: Oops. Not a Vary match on 
second attempt, 'http://s7.addthis.com/js/250/addthis_widget.js' 
'accept-encoding="gzip,deflate"'
2015/09/02 15:03:59 kid1| clientProcessHit: Vary object loop!



Later on I tested it with this short config file and the problem persisted:

http_access allow localhost manager
http_access deny manager
acl purge method PURGE
http_access allow purge localhost
http_access deny purge
acl all src all
acl localhost src 127.0.0.1/32
acl localnet src 127.0.0.0/8
acl Safe_ports port 80
acl snmppublic snmp_community public
http_access deny !Safe_ports
http_access allow all
dns_v4_first on
cache_mem 1024 MB
maximum_object_size_in_memory 64 KB
memory_cache_mode always
maximum_object_size 150000 KB
minimum_object_size 100 bytes
collapsed_forwarding on
logfile_rotate 5
mime_table /etc/squid3/mime.conf
debug_options ALL,1
store_id_access deny all
store_id_bypass on
refresh_pattern ^ftp:                    1440    20%    10080
refresh_pattern ^gopher:                1440    0%    1440
refresh_pattern ^http:\/\/movies\.apple\.com           86400 20%     
86400 override-expire override-lastmod ignore-no-cache ignore-private 
ignore-reload
refresh_pattern -i \.flv$                   10080   90%     999999 
ignore-no-cache override-expire ignore-private
refresh_pattern -i \.mov$                   10080   90%     999999 
ignore-no-cache override-expire ignore-private
refresh_pattern windowsupdate.com/.*\.(cab|exe) 4320 100% 43200 
reload-into-ims
refresh_pattern download.microsoft.com/.*\.(cab|exe) 4320 100% 43200 
reload-into-ims
refresh_pattern -i 
\.(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|pdf|tiff)$ 10080 90% 
43200 override-expire ignore-no-cache ignore-private
refresh_pattern -i (/cgi-bin/)             0    0%    0
refresh_pattern .                    0    20%    4320
quick_abort_min 0 KB
quick_abort_max 0 KB
quick_abort_pct 100
range_offset_limit 0
negative_ttl 1 minute
negative_dns_ttl 1 minute
read_ahead_gap 128 KB
request_header_max_size 100 KB
reply_header_max_size 100 KB
via off
acl apache rep_header Server ^Apache
half_closed_clients off
cache_mgr webmaster
cache_effective_user squid
cache_effective_group squid
httpd_suppress_version_string on
snmp_access allow snmppublic localhost
snmp_access deny all
snmp_incoming_address 127.0.0.1
error_directory /etc/squid3/errors/English
max_filedescriptors 65535
ipcache_size 1024
forwarded_for off
log_icp_queries off
icp_access allow localnet
icp_access deny all
htcp_access allow localnet
htcp_access deny all
digest_rebuild_period 15 minutes
digest_rewrite_period 15 minutes
strip_query_terms off
max_open_disk_fds 150
cache_replacement_policy heap LFUDA
memory_pools off
http_port 9001
http_port 901 tproxy
if ${process_number} = 1
access_log stdio:/var/log/squid/1/access.log squid
cache_log /var/log/squid/1/cache.log
cache_store_log none
cache_swap_state /var/log/squid/1/%s.swap.state
else
  access_log none
  cache_log /dev/null
endif
pid_filename /var/run/squid1.pid
visible_hostname localhost
snmp_port 1611
icp_port 3131
htcp_port 4828
cachemgr_passwd admin thisisnotmyrealpassword
memory_cache_shared  off
cache_dir rock  /cache1/rock1 256  min-size=100 max-size=3000
cache_dir rock  /cache1/rock2 2000  min-size=3000 max-size=20000
cache_dir diskd /cache1/diskd2 60000 16 256 min-size=20000 max-size=200000
cache_dir diskd /cache2/2 100000 16 256 min-size=200000 max-size=1048576
cache_dir diskd /cache2/1 680000 16 256 min-size=1048576



Any ideas what could be wrong?



Thanks,
Sebastian






El 26/08/15 a las 17:15, Amos Jeffries escribió:
> On 27/08/2015 7:53 a.m., Sebastián Goicochea wrote:
>> After I sent you my previous email, I continued investigating the
>> subject .. I made a change in the source code as follows:
>>
>> File: /src/http.cc
>>
>> HttpStateData::haveParsedReplyHeaders()
>> {
>>      .
>>      .
>> ##### THIS IS NEW STUFF ###########
>>      if (rep->header.has(HDR_VARY)) {
>>      rep->header.delById(HDR_VARY);
>>      debugs(11,3, "Vary detected. Hack Cleaning it up");
>>      }
>> ##### END OF NEW STUFF ###########
>>
>> #if X_ACCELERATOR_VARY
>>      if (rep->header.has(HDR_X_ACCELERATOR_VARY)) {
>>      rep->header.delById(HDR_X_ACCELERATOR_VARY);
>>      debugs(11,3, "HDR_X_ACCELERATOR_VARY Vary detected. Hack Cleaning it
>> up");
>>      }
>> #endif
>>      .
>>      .
>>
>>
>> Deleting Vary from the header at this point gives me hits in every
>> object I test (that previously didn't hit) .. web browser never receives
>> the Vary in the response header.
>> Now I read your answer and you say that this is a critical validity
>> check and that worries me. Taking away the vary altogether at this point
>> could lead to the problems that you described? If that is the case .. I
>> have to investigate other alternatives.
>>
> I'll have to look into that function when I'm back at the code later to
> confirm this. But IIRC that function is acting directly on a freshly
> received reply message. You are not removing the validity check, you are
> removing Squids ability to see that it is a Vary object at all. So it is
> never even cached as one.
>
> The side effect of that is that clients asking for non-gzip can get the
> cached gzip copy, etc. but at least its the same URL. So the security
> risks are gone. But the user experience is not always good either way.
>
> Amos
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20150903/4f1474a6/attachment.html>


More information about the squid-users mailing list