[squid-dev] The next step towards: StoreID and metalink.

[Amos Jeffries]

On 22/09/2015 6:19 a.m., Alex Rousskov wrote:
> On 09/19/2015 05:28 PM, Eliezer Croitoru wrote:
> 
>> There are things which can help an ECAP module to decide how to do
>> things when handling metalinks related affairs.
>> For example, a cache object lookup from within an ECAP module.
>> So basically the logic of the ECAP module is 100% blind to the cache
>> internals.
> 
> FWIW, there have been requests to expose host cache state to eCAP
> adapters. It can be easily done (via Cache-Lookup metadata) for the
> current request [in pre-cache REQMOD]. Doing so will even be compatible
> with ICAP services.
> 
> Unfortunately, but most real use cases I know about actually need a true
> "cache lookup using an arbitrary request and/or StoreID" functionality
> and not just a "is the current request likely to be served from cache?"
> answer. Supporting arbitrary lookups would require expanding eCAP API. I
> would support such expansion, especially if we have several different
> specific use cases the back the new API design up.
> 
> Alex.
> 

What I have been thinking of for Metalinks was a StoreID helper that
took the requested URI and matched it against an index/DB of it own to
see if there was a previous ID for it.

That could be paired with an eCAP RESPMOD adapter that scanned for
metalink URIs and added records to the StoreID index/DB for them. So
that future requests for those URIs got the now-cached content.

There are issues around invalidation to work out. Specifically when the
cached object gets replaced and has different metalinks attached, how to
find the obsolete ones. Doing a delete operation at speed may be
problematic.

If the index/DB could be Squids internal cache index, or a special case
per-cache indexes like Transients but for the eCAP adapter. Then the
StoreID helper may not be even necessary. But that might be a lot more work.

Amos