[squid-users] Cache reference age for heap LRU/LFUDA and rock/aufs

[Ivan Larionov]

On Fri, Feb 9, 2018 at 7:50 PM, Alex Rousskov <
rousskov at measurement-factory.com> wrote:

>
> I cannot answer your question for aufs, but please note that rock
> cache_dirs do not support/have/use a configurable replacement policy:
> Each incoming object is assigned a slot based on its key hash. With
> modern rock code, it is possible to remove that limitation IIRC, but
> nobody have done that.
>

Yeah I figured this out from the source code and I'm extremely surprised by
the fact that it was never mentioned in documentation. I think it will be a
huge blocker in our squid 4 + SMP + rock migration plan.

So what does rock do when storage is full then?


>
>
> > If you're wondering why would we need to know that – it's related to
> > GDPR and removing data of closed customer's accounts. We need to make
> > sure that we don't have any "not being accessed anymore" objects older
> > than "data retention period" days.
>
> If it is important to get this right, then I would not trust replacement
> policy metadata with this: The corresponding code interfaces look
> unreliable to me, and access counts/timestamps for a ufs-based cache_dir
> are not updated across Squid restarts when the swap log is lost (at least).
>
>
It's actually fine, we never restart squid and if it restarted by any
unexpected reason (host reboot, crash or w/e) we just replace the host.


> I would instead configure Squid to prohibit serving hits that are too
> old. That solution does not match your problem exactly, but it may be
> good enough and should work a lot more reliably across all cache_dirs.
> If there is no "age" ACL to use with the send_hit directive, then you
> may need to add one.
>
>     http://www.squid-cache.org/Doc/config/send_hit/
>
> You may also be able to accomplish the same using refresh_pattern, but I
> am a little worroed about various exceptional/special conditions
> implemented on top of that directive. Others on this list may offer
> better guidance in this area.
>
>
I was thinking about similar solution but this is exactly why I wasn't able
to use it – there seems to be no acl suitable for such task.

We can always just replace the host every month or something like this but
it'll mean starting with a cold cache every time which I wanted to avoid.

I found this debug option for heap which could probably help in
understanding of approximate cache age but it doesn't work with rock
because rock uses some "simple scan" policy.

> src/repl/heap/store_repl_heap.cc:        debugs(81, 3, "Heap age set to "
<< h->theHeap->age);


> HTH,
>
> Alex.
>



-- 
With best regards, Ivan Larionov.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20180212/017ac570/attachment-0001.html>