[squid-users] Squid issue of caching the m3u8 file

[Amos Jeffries]

On 05/06/17 23:54, LIU Yaning wrote:
> Dear All,
>
> I would like to cache the .m3u8 file to be able to provide offline 
> caching service by Squid. The played HLS video streaming is the link 
> as below:
> http://qthttp.apple.com.edgesuite.net/1010qwoeiuryfg/0150_vod.m3u8 
> <http://qthttp.apple.com.edgesuite.net/1010qwoeiuryfg/0150_vod.m3u8>
> However, the .m3u8 file is not be cached probably because it is 
> mentioned as a no-cache, no-store, max-age=0 in the "Cache-Control" in 
> the HTTP header.
Nope. Only the CC:no-store is preventing caching. The other headers 
simply put boundaries on what is to be done with the content in the 
cache. In particular the "no-cache", max-age=0 and Expires values mean 
it has to be revalidated (REFRESH in your access.log) before any future 
uses - probably because of that Set-Cookie needing to be changed for 
different end-users.
>
> HTTP/1.1 200 OK
>
> Server: Apache
>
> ETag: "1d7168b4f49e75f76f3182f24bf075f6:1299516751"
>
> Last-Modified: Mon, 07 Mar 2011 16:52:31 GMT
>
> Expires: Fri, 02 Jun 2017 14:26:52 GMT
>
> Cache-Control: max-age=0, no-cache, no-store
>
> Pragma: no-cache
>
> Date: Fri, 02 Jun 2017 14:26:52 GMT
>
> Content-Length: 16046
>
> Set-Cookie: AKID=77F9F1316ECCE780566608C5E514DE0A;expires=Fri, 26 Aug 
> 2016 00:01:00 GMT; path=/; domain=qthttp.apple.com.edgesuite.net 
> <http://qthttp.apple.com.edgesuite.net/>
>
> Content-Type: application/x-mpegURL
>
> Access-Control-Allow-Origin: *
>
> I added a new rule for .m3u8 file in squid.conf, however, it is still 
> not working.
>
> refresh_pattern -i \.(ts|m3u8)$ 120 90% 1000 override-expire 
> override-lastmod ignore-no-cache ignore-no-store
>
> Does anyone know how to allow Squid caching the .m3u8 file? Thanks a 
> lot in advance.
What makes you think it is not caching? The ignore-no-store alone should 
be sufficient to allow current Squid versions to cache that object. You 
could perhapse add "store-stale" option on that config line. Which 
should make Squid cache object containing an Expires header with current 
or past values. refresh_pattern settings do not affect that cacheable vs 
non-cacheable decision. The no-cache header tells Squid the object needs 
revalidating before every use. However, be aware the tool at redbot.org 
tells me that this URL is badly broken in how it is using the ETag and 
Vary headers - in a way which can break the revalidation when these 
things are cached. Some of your clients may see very broken behaviour 
accessing this object unless you follow the no-store requirement or the 
server stops its broken ETag behaviour. PS. if you are using a Squid 
version much older than 3.5.24 I recommend an upgrade. With an urgency 
increasing the older your Squid is. Amos