[squid-users] squid doesn't cache objects in memory when using SMP and shared memory cache

[Ivan Larionov]

Yeah I meant Very in response to requests I tried.

The fact that you can't reproduce it with this url and the fact that it
affects mostly mp3/wav files gave me an idea.

Our env specific is squid's cache_peer parent which transcodes audio files
to ulaw (format which our backend supports). This explains why audio files.
Doesn't explain why it works with squid 3 non-shared memory and squid 4
shared-memory.

I verified direct request and memory cache works with squid 3 +shared
memory.

The difference between direct and non-direct (transcoded) response for
http://techslides.com/demos/samples/sample.mp3:

* "Content-Type: audio/mpeg" for direct, "Content-Type: audio/ulaw" for
non-direct.
* No "Content-Length" header for non-direct.

What do you think? Could these 2 points lead to the issue I see? Why does
it work in all situations except squid 3 + shared memory?

On Tue, Jan 16, 2018 at 3:17 PM, Alex Rousskov <
rousskov at measurement-factory.com> wrote:

> On 01/16/2018 02:40 PM, Ivan Larionov wrote:
> > So it's definitely not related to Vary, there's no such header in
> > requests I tried.
>
> Just to avoid misunderstanding, please note that Vary is a response header.
>
>
> > Also this issue affects squid even with 1 worker if
> > shared memory is forced to on.
>
> This matches the current suspicion that there is something wrong with
> the shared memory cache (in your environment).
>
> > It's not like it makes any sense but how about random example url:
> >
> > http://techslides.com/demos/samples/sample.mp3
> >
> > squid 4, 2 workers, shared cache, no disk cache – MEM_HIT
> > squid 3, same config – MISS every time
> > squid 3, no shared cache – MEM_HIT
> >
> > Could you do a brief test with this URL may be and confirm that I'm not
> > the only one who see this issue?
>
> I cannot confirm that: In primitive wget tests with Squid v3.5 (bzr
> r14182), I am getting shared memory hits with the above URL, with or
> without cache_dirs, with or without SMP.
>
> Alex.
>
>
> > On Tue, Jan 16, 2018 at 6:25 AM, Alex Rousskov wrote:
> >
> >     On 01/15/2018 09:25 PM, Ivan Larionov wrote:
> >     > My total hit ratio decreased in ~2 times from 40% to 20%
> >
> >     >     On 01/14/2018 10:53 PM, Ivan Larionov wrote:
> >     >
> >     >     > After migrating squid from non-SMP/aufs to SMP/rock memory
> cache hit
> >     >     > ratio dropped significantly. Like from 50-100% to 1-5%.
> >     >
> >     >     > And disk cache hit ratio went up from 15-50% to stable
> 60-65%.
> >
> >
> >     The combination of the three statements above may be a sign of a
> problem
> >     unrelated to Vary: Since the disk cache can cache everything the
> memory
> >     cache can and is typically much larger than the memory cache, the
> >     incapacitation of a memory cache (due to Vary) should not have a
> >     significant effect on overall hit ratio. It should only affect hit
> >     response time.
> >
> >     The only known culprit I can think of in this context are hits for
> >     being-cached objects: Rock lacks code that allows Squid to read
> >     being-written objects. The shared memory cache has that code
> already. If
> >     your workload has a lot of cases where clients request a
> being-written
> >     object, then the overall hit ratio should go down after the memory
> cache
> >     incapacitation (due to Vary).
> >
> >     I suspect something else is in play here though, especially if you
> see a
> >     different picture with Squid v4 -- the known problem discussed above
> is
> >     present in all Squid versions. I second Amos's recommendation to
> focus
> >     on v4 because it is unlikely that any complicated problems are going
> to
> >     be fixed in v3, even if you triage them well.
> >
> >
> >     HTH,
> >
> >     Alex.
> >
> >
> >
> >
> > --
> > With best regards, Ivan Larionov.
>
>


-- 
With best regards, Ivan Larionov.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20180116/1d620fde/attachment-0001.html>