[squid-users] Squid downloading huge amounts of un-requested data

Thu Apr 16 03:16:55 UTC 2015

Hi Forum,

We're running Squid Version 3.3.8 (from repository) on Ubuntu server 14.04
LTS x64 as a caching proxy server for a mixed Linux/Windows network. 
We were attempting to cache apt-get and Windows updates using various
refresh patterns but commented them out a few weeks ago after an issue where
the squid server had downloaded over 100GB of data over 24 hours but in that
time had served only about 6GB to clients. 100GB represents almost our
entire normal monthly traffic. Most of the incoming Squid traffic was from
Windows updates or Akamai servers. Prior to that the server had been running
for months without an issue.
Unfortunately the same thing happened again last night. The only remanent of
the Windows Update cache commands we forgot to uncomment in squid.conf were:
range_offset_limit 200 MB WindowsUpdate 
maximum_object_size 1 GB
quick_abort_min -1

which we have since commented out and restarted the Squid server.

The incoming traffic to Squid is being recorded by the syslog of our ASA5510
firewall and corroborated by our ISP's management console. Clients can only
access the internet through the proxy. The outgoing traffic statistics are
being reported by SARG, SquidAnalyzer and a custom tcpdump script of all
traffic from the server and all three agree.

So our question- is there any way that Squid can download content
repeatedly, and many times in excess of what our clients are actually
requesting? How can we debug this issue further? 

Grateful for any advice you

some squidclient output:
Squid Object Cache: Version 3.3.8
Start Time:     Wed, 15 Apr 2015 23:11:20 GMT
Current Time:   Thu, 16 Apr 2015 03:14:41 GMT
Connection information for squid:
        Number of clients accessing cache:      46
        Number of HTTP requests received:       42126
        Number of ICP messages received:        0
        Number of ICP messages sent:    0
        Number of queued ICP replies:   0
        Number of HTCP messages received:       0
        Number of HTCP messages sent:   0
        Request failure ratio:   0.00
        Average HTTP requests per minute since start:   173.1
        Average ICP messages per minute since start:    0.0
        Select loop called: 5688640 times, 2.567 ms avg
Cache information for squid:
        Hits as % of all requests:      5min: 17.9%, 60min: 11.3%
        Hits as % of bytes sent:        5min: 4.2%, 60min: 7.2%
        Memory hits as % of hit requests:       5min: 73.5%, 60min: 63.0%
        Disk hits as % of hit requests: 5min: 0.0%, 60min: 17.2%
        Storage Swap size:      11278780 KB
        Storage Swap capacity:  44.1% used, 55.9% free
        Storage Mem size:       153892 KB
        Storage Mem capacity:    7.5% used, 92.5% free
        Mean Object Size:       23.21 KB
        Requests given to unlinkd:      0
Median Service Times (seconds)  5 min    60 min:
        HTTP Requests (All):   0.00000  0.03241
        Cache Misses:          0.46965  0.18699
        Cache Hits:            0.00000  0.00091
        Near Hits:             0.04776  0.05331
        Not-Modified Replies:  0.01745  0.00091
        DNS Lookups:           0.01609  0.01609
        ICP Queries:           0.00000  0.00000

 relevant squid.conf

cache_mem 2000 MB
maximum_object_size_in_memory 1 MB

# Log file locations
access_log daemon:/var/log/squid3/access.log squid
cache_store_log none
cache_log /var/log/squid3/cache.log
coredump_dir /var/spool/squid3

# Disk cache directory.
cache_dir aufs /squid_cache/Squid3Cache 25000 16 256refresh_pattern ^ftp:          
1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern .               0       20%     4320

dns_nameservers 127.0.0.1

icap_enable on
icap_send_client_ip on
icap_send_client_username on
icap_client_username_encode off
icap_client_username_header X-Authenticated-User
icap_preview_enable on
icap_preview_size 1024
icap_service service_req reqmod_precache bypass=0
icap://127.0.0.1:1344/squidclamav
adaptation_access service_req allow all
icap_service service_resp respmod_precache bypass=0
icap://127.0.0.1:1344/squidclamav
adaptation_access service_resp allow all

url_rewrite_program /usr/bin/squidGuard -c /etc/squidguard/squidGuard.conf
url_rewrite_children 20 startup=0 idle=1 concurrency=0

--
View this message in context: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-downloading-huge-amounts-of-un-requested-data-tp4670770.html
Sent from the Squid - Users mailing list archive at Nabble.com.