[squid-users] Possible bug in 3.5.5 or a store change from 2.7?

[Amos Jeffries]

On 29/07/2015 10:43 a.m., Tory M Blue wrote:
> On Tue, Jul 28, 2015 at 12:54 PM, Amos Jeffries <squid3 at treenet.co.nz>
> wrote:
> 
>> On 29/07/2015 5:53 a.m., Tory M Blue wrote:
>>
>>
>>>  I just reproduced this by hand, using an HTTP sniffer tool.  I requested
>>> the same URL twice, with about a 0.25 second delay between fetches, and
>> the
>>> 2nd attempt was ALSO A MISS.  Then I waited 1 second and tried a 3rd
>> time,
>>> and it was FINALLY a hit.
>>
>> Had the first request finished being delivered to your testing tool
>> before the second request was sent?
>>
> 
> 
> Yes our testing is completely sequential meaning the connection has to
> complete before we launch another, even by hand.
> 
> 
> 
>>
>>> Squid v3 seems to have changed the way it stores objects.  Maybe it is
>>> doing some sort of "asynchronous" background store now, so if you send in
>>> sequential requests without a delay between, it may not actually have
>>> finished storing it yet, so it doesn't report a "hit".  Meaning, the
>> first
>>> "miss" response may have fired off a thread to store the object, and not
>>> doing it in the main thread anymore like in v2, if you get my meaning.
>>
>> Squid-3 in-transit objects are not listed as existing in cache_mem
>> storage until they have completely (and successfully) finished their
>> first use.
>>
>>
>> You can configure "collapsed_forwarding on" to enable the Squid-2.7
>> behaviour. Treating the in-transit objects list as if it were another
>> cache area.
>>
>>
>> The document cite this is quite a bit different and not something I want
> to embark upon.
> 
>>
>>>
>>
>>
>> Timing between *completion* of the first request and starting of the
>> second is the critical. If the first request has not finished, theres no
>> hope of a HIT.
>>
>> Also size of memory cache relative to the churn currently going on also
>> matters. If you wait too long between the test requests it will be
>> pushed out and get a MISS again anyway. But I dont think that is a
>> factor with 250ms being your timing.
>>
> 
> 
> Again completely synchronous, something is not happening as it should under
> a busy condition, we tested this as far out as 5 seconds and still got a
> miss.  We have added "5" retries to our monitor, but I do think something
> is  a miss (no pun intended).  Single client hitting a single server in a
> synchronous manner, should allow for a pretty quick cache hit, but it's
> not.
> 
> my high/low water marks are so
> 
> cache_swap_high 90
> 
> cache_swap_low 80
> 
> 
> My space available is so
> 
> Filesystem            Size  Used Avail Use% Mounted on
> 
> /dev/ram0              17G   14G  2.3G  86% /cache01
> 
> Any other data I could grab from the stats that would help. I do believe
> something is not quite right but we have ways to get around it for now,
> 

That depends on how busy. A debug trace at level ALL,7 would be best.
But the amount of date logged can be a problem if Squid is running
several hundred or thousand req/sec.

That can be resolved somewhat by turning the logging on dynamically with
"squid -k debug" shortly before and after a test is run.

Amos