[squid-users] Squid transfers much not requested data from uplink in specific cases

[Garri Djavadyan]

Hello Squid community,

According to the bug report 4511 [1], Squid may transfer much useless,
not requested data from uplink after specific sequence of actions.

For example, slow client (access rate 128Kb/s) may begin transfer of
big cacheable object (4GB). After some time, another client (access
rate 1Mb/s) may begin to transfer the same object. Depending on time
interval between the requests (or the volume to data cached to disk by
the first transfer), after some time, transfer rate of the second
client will be limited to 128Kb/s too [2]. If after some time the first
client aborts the transfer, the second client's rate recovers to 1Mb/s
and Squid begin to transfer the object from origin server to disk cache
at maximum rate the uplink permits (let's assume 100Mb/s).

If the second client also aborts the transfer after some time, it may
results in much not requested data transferred from uplink. Also, if
Squid doesn't finish to transfer the object to the disk cache, the
object won't be cached finally.

For example, if the first client aborts the transfer after downloading
100MB, Squid begins to transfer the object at rate 100Mb/s. If after 4
minutes, the second client also aborts the transfer, Squid would
terminate the transfer from origin server and we get is the following
statistics:

Client #1 - 100MB
Client #2 - 100MB + 1Mb/s / 8 bits * 240 seconds = 130MB
Squid - 100MB + 100Mb/s / 8 bits * 240 seconds = 3100MB
Not requested / useless data = 3100MB - (100MB + 130MB) = 2870MB

In that scenario, Squid would get 2870MB useless data that would not be
cached (object's size 4GB). During these 4 minutes, 'Hits as % bytes
sent' counter would show extremely low value.

[1] http://bugs.squid-cache.org/show_bug.cgi?id=4511
[2] http://bugs.squid-cache.org/show_bug.cgi?id=4520

------------

So, I want to ask community to share ideas, best practice to cope with
the problem. Many thank in advance!

-- 
Garri Djavadyan <garryd at comnet.uz>
Comnet ISP