[squid-users] Squid downloading huge amounts of un-requested data

[Amos Jeffries]

On 16/04/2015 3:16 p.m., iridium191 wrote:
> Hi Forum,
> 
> We're running Squid Version 3.3.8 (from repository) on Ubuntu server 14.04
> LTS x64 as a caching proxy server for a mixed Linux/Windows network. 
> We were attempting to cache apt-get and Windows updates using various
> refresh patterns but commented them out a few weeks ago after an issue where
> the squid server had downloaded over 100GB of data over 24 hours but in that
> time had served only about 6GB to clients. 100GB represents almost our
> entire normal monthly traffic. Most of the incoming Squid traffic was from
> Windows updates or Akamai servers. Prior to that the server had been running
> for months without an issue.
> Unfortunately the same thing happened again last night. The only remanent of
> the Windows Update cache commands we forgot to uncomment in squid.conf were:
> range_offset_limit 200 MB WindowsUpdate 
> maximum_object_size 1 GB
> quick_abort_min -1
> 
> which we have since commented out and restarted the Squid server.

Leave the maximum_object_size, the default is to only cache 4 MB objects
and anything else becomes a MISS to consume more upstream bandwidth.

The range_offset_limit and quick_abort_min are what causes more download
for Range requests than what clients requested.


> 
> The incoming traffic to Squid is being recorded by the syslog of our ASA5510
> firewall and corroborated by our ISP's management console. Clients can only
> access the internet through the proxy. The outgoing traffic statistics are
> being reported by SARG, SquidAnalyzer and a custom tcpdump script of all
> traffic from the server and all three agree.
> 
> So our question- is there any way that Squid can download content
> repeatedly, and many times in excess of what our clients are actually
> requesting? How can we debug this issue further? 


Several.

* Remote server not supporting HTTP/1.1 304 revalidation properly.
Results in the server delivering 200 response + full object data to
Squid when a 304 would suffice and Squid only delivers 304 to the client.

* Misconfigured / open proxy. When remote users abuse the proxy all
traffic goes over the WAN interfaces. Measurement tools that
differentiate by NIC on the Squid machine will see this as massive
amounts of WAN traffic relative to LAN.

* Long polling connections.

* ICAP. The ICAP service may be doing anything that triggers lots of WAN
traffic use. It can require full objects - forcing the same behaviour as
quick_abort_*, it can do its own downloads - traffic not used by Squid
at all but still used, it can make lots of requests

* squidguard. Same as with ICAP it can be doing its own downloads. Also
URL-rewritten traffic does not relay the client reuqest to the server,
but the re-writer generated request. This is another way server 200
responses can become 200 when only 304 were necessary.



>  relevant squid.conf

Is missing many configuration details that must be present for an
operational proxy. Specifically the access controls to determine if the
open proxy issues are relevant.


With all that logging and traffic measurement are you able to track down
an example transaction where Squid fetches more from the network than it
delivers to the client.

Squids cachemgr utilization report has a breakdown of how much traffic
is used by each I/O type. It can help identify where to look at closer.


Amos