[squid-users] Lots of "Vary object loop!"
Kinkie
gkinkie at gmail.com
Thu Sep 3 16:37:06 UTC 2015
Hi,
do you think you could manage to capture the headers of the
response triggering that error?
I've been looking that up, but couldn't reprduce it.
The good news is, it's mostly harmless: worst case scenario it will
cause a slow cache miss.
Thanks
On Thu, Sep 3, 2015 at 5:20 PM, Sebastián Goicochea
<sebag at vianetcon.com.ar> wrote:
> Amos, I spent a couple of days doing some test with the info you gave me:
>
> Retested emptying the cache several times, disabled the rewriter, different
> config files .. all I could think of
>
>
> Downloaded fresh 3.5.8 tar.gz (just in case it was some 3.5.4 thing) and
> compiled it using this configure options:
>
> Squid Cache: Version 3.5.8
> Service Name: squid
> configure options: '--prefix=/usr/local' '--datadir=/usr/local/share'
> '--bindir=/usr/local/sbin' '--libexecdir=/usr/local/lib/squid'
> '--localstatedir=/var' '--sysconfdir=/etc/squid3' '--enable-delay-pools'
> '--enable-ssl' '--enable-ssl-crtd' '--enable-linux-netfilter' '--enable-eui'
> '--enable-snmp' '--enable-gnuregex' '--enable-ltdl-convenience'
> '--enable-removal-policies=lru heap' '--enable-http-violations'
> '--with-openssl' '--with-filedescriptors=24321' '--enable-poll'
> '--enable-epoll' '--enable-storeio=ufs,aufs,diskd,rock' '--disable-ipv6'
>
>
>
> And the problem appeared again, I am suspicious that the problem is in the
> configuration, I even removed all my refresh patterns, but:
>
> 2015/09/02 15:03:42 kid1| varyEvaluateMatch: Oops. Not a Vary match on
> second attempt, 'http://assets.pinterest.com/js/pinit.js'
> 'accept-encoding="gzip,%20deflate"'
> 2015/09/02 15:03:42 kid1| clientProcessHit: Vary object loop!
> 2015/09/02 15:03:43 kid1| varyEvaluateMatch: Oops. Not a Vary match on
> second attempt, 'http://static.cmptch.com/v/lib/str.html'
> 'accept-encoding="gzip,%20deflate,%20sdch"'
> 2015/09/02 15:03:43 kid1| clientProcessHit: Vary object loop!
> 2015/09/02 15:03:43 kid1| varyEvaluateMatch: Oops. Not a Vary match on
> second attempt,
> 'http://pstatic.bestpriceninja.com/nwp/v0_0_773/release/Shared/Extra/IFrameStoreReciever.js'
> 'accept-encoding="gzip,%20deflate,%20sdch"'
> 2015/09/02 15:03:43 kid1| clientProcessHit: Vary object loop!
> 2015/09/02 15:03:59 kid1| varyEvaluateMatch: Oops. Not a Vary match on
> second attempt, 'http://static.xvideos.com/v2/css/xv-video-styles.css?v=7'
> 'accept-encoding="gzip,deflate"'
> 2015/09/02 15:03:59 kid1| clientProcessHit: Vary object loop!
> 2015/09/02 15:03:59 kid1| varyEvaluateMatch: Oops. Not a Vary match on
> second attempt, 'http://s7.addthis.com/js/250/addthis_widget.js'
> 'accept-encoding="gzip,deflate"'
> 2015/09/02 15:03:59 kid1| clientProcessHit: Vary object loop!
>
>
>
> Later on I tested it with this short config file and the problem persisted:
>
> http_access allow localhost manager
> http_access deny manager
> acl purge method PURGE
> http_access allow purge localhost
> http_access deny purge
> acl all src all
> acl localhost src 127.0.0.1/32
> acl localnet src 127.0.0.0/8
> acl Safe_ports port 80
> acl snmppublic snmp_community public
> http_access deny !Safe_ports
> http_access allow all
> dns_v4_first on
> cache_mem 1024 MB
> maximum_object_size_in_memory 64 KB
> memory_cache_mode always
> maximum_object_size 150000 KB
> minimum_object_size 100 bytes
> collapsed_forwarding on
> logfile_rotate 5
> mime_table /etc/squid3/mime.conf
> debug_options ALL,1
> store_id_access deny all
> store_id_bypass on
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern ^http:\/\/movies\.apple\.com 86400 20% 86400
> override-expire override-lastmod ignore-no-cache ignore-private
> ignore-reload
> refresh_pattern -i \.flv$ 10080 90% 999999
> ignore-no-cache override-expire ignore-private
> refresh_pattern -i \.mov$ 10080 90% 999999
> ignore-no-cache override-expire ignore-private
> refresh_pattern windowsupdate.com/.*\.(cab|exe) 4320 100% 43200
> reload-into-ims
> refresh_pattern download.microsoft.com/.*\.(cab|exe) 4320 100% 43200
> reload-into-ims
> refresh_pattern -i \.(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|pdf|tiff)$
> 10080 90% 43200 override-expire ignore-no-cache ignore-private
> refresh_pattern -i (/cgi-bin/) 0 0% 0
> refresh_pattern . 0 20% 4320
> quick_abort_min 0 KB
> quick_abort_max 0 KB
> quick_abort_pct 100
> range_offset_limit 0
> negative_ttl 1 minute
> negative_dns_ttl 1 minute
> read_ahead_gap 128 KB
> request_header_max_size 100 KB
> reply_header_max_size 100 KB
> via off
> acl apache rep_header Server ^Apache
> half_closed_clients off
> cache_mgr webmaster
> cache_effective_user squid
> cache_effective_group squid
> httpd_suppress_version_string on
> snmp_access allow snmppublic localhost
> snmp_access deny all
> snmp_incoming_address 127.0.0.1
> error_directory /etc/squid3/errors/English
> max_filedescriptors 65535
> ipcache_size 1024
> forwarded_for off
> log_icp_queries off
> icp_access allow localnet
> icp_access deny all
> htcp_access allow localnet
> htcp_access deny all
> digest_rebuild_period 15 minutes
> digest_rewrite_period 15 minutes
> strip_query_terms off
> max_open_disk_fds 150
> cache_replacement_policy heap LFUDA
> memory_pools off
> http_port 9001
> http_port 901 tproxy
> if ${process_number} = 1
> access_log stdio:/var/log/squid/1/access.log squid
> cache_log /var/log/squid/1/cache.log
> cache_store_log none
> cache_swap_state /var/log/squid/1/%s.swap.state
> else
> access_log none
> cache_log /dev/null
> endif
> pid_filename /var/run/squid1.pid
> visible_hostname localhost
> snmp_port 1611
> icp_port 3131
> htcp_port 4828
> cachemgr_passwd admin thisisnotmyrealpassword
> memory_cache_shared off
> cache_dir rock /cache1/rock1 256 min-size=100 max-size=3000
> cache_dir rock /cache1/rock2 2000 min-size=3000 max-size=20000
> cache_dir diskd /cache1/diskd2 60000 16 256 min-size=20000 max-size=200000
> cache_dir diskd /cache2/2 100000 16 256 min-size=200000 max-size=1048576
> cache_dir diskd /cache2/1 680000 16 256 min-size=1048576
>
>
>
> Any ideas what could be wrong?
>
>
>
> Thanks,
> Sebastian
>
>
>
>
>
>
> El 26/08/15 a las 17:15, Amos Jeffries escribió:
>
> On 27/08/2015 7:53 a.m., Sebastián Goicochea wrote:
>
> After I sent you my previous email, I continued investigating the
> subject .. I made a change in the source code as follows:
>
> File: /src/http.cc
>
> HttpStateData::haveParsedReplyHeaders()
> {
> .
> .
> ##### THIS IS NEW STUFF ###########
> if (rep->header.has(HDR_VARY)) {
> rep->header.delById(HDR_VARY);
> debugs(11,3, "Vary detected. Hack Cleaning it up");
> }
> ##### END OF NEW STUFF ###########
>
> #if X_ACCELERATOR_VARY
> if (rep->header.has(HDR_X_ACCELERATOR_VARY)) {
> rep->header.delById(HDR_X_ACCELERATOR_VARY);
> debugs(11,3, "HDR_X_ACCELERATOR_VARY Vary detected. Hack Cleaning it
> up");
> }
> #endif
> .
> .
>
>
> Deleting Vary from the header at this point gives me hits in every
> object I test (that previously didn't hit) .. web browser never receives
> the Vary in the response header.
> Now I read your answer and you say that this is a critical validity
> check and that worries me. Taking away the vary altogether at this point
> could lead to the problems that you described? If that is the case .. I
> have to investigate other alternatives.
>
> I'll have to look into that function when I'm back at the code later to
> confirm this. But IIRC that function is acting directly on a freshly
> received reply message. You are not removing the validity check, you are
> removing Squids ability to see that it is a Vary object at all. So it is
> never even cached as one.
>
> The side effect of that is that clients asking for non-gzip can get the
> cached gzip copy, etc. but at least its the same URL. So the security
> risks are gone. But the user experience is not always good either way.
>
> Amos
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
>
>
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
--
Francesco
More information about the squid-users
mailing list