[squid-users] Squid issue of caching the m3u8 file

Tue Jun 6 13:33:22 UTC 2017

Dear Amos,

Thanks a lot for your explanation and suggestion. I added the "store-stale"
to the refresh_pattern rule as:
refresh_pattern -i \.(ts|m3u8)$ 120 90% 1000 override-expire override-lastmod
ignore-no-cache ignore-no-store store-stale

However, I have checked the access.log, I am still getting TCP_Miss.
1496754869.963 13 192.168.0.100 TCP_MISS/200 16636 GET
http://qthttp.apple.com.edgesuite.net/1010qwoeiuryfg/0150_vod.m3u8
<http://www.google.com/url?q=http%3A%2F%2Fqthttp.apple.com.edgesuite.net%2F1010qwoeiuryfg%2F0150_vod.m3u8&sa=D&sntz=1&usg=AFQjCNHpiHy55EMeBIaMGhgEKRHanTrXxg>
-
HIER_DIRECT/95.101.182.201
<http://www.google.com/url?q=http%3A%2F%2F95.101.182.201&sa=D&sntz=1&usg=AFQjCNE9ZWXH7sOJgbqIA--MJwxobSp76Q>
 application/x-mpegURL
1496754870.605 4 192.168.0.100 TCP_MISS/200 16636 GET
http://qthttp.apple.com.edgesuite.net/1010qwoeiuryfg/0150_vod.m3u8
<http://www.google.com/url?q=http%3A%2F%2Fqthttp.apple.com.edgesuite.net%2F1010qwoeiuryfg%2F0150_vod.m3u8&sa=D&sntz=1&usg=AFQjCNHpiHy55EMeBIaMGhgEKRHanTrXxg>
-
HIER_DIRECT/95.101.182.201
<http://www.google.com/url?q=http%3A%2F%2F95.101.182.201&sa=D&sntz=1&usg=AFQjCNE9ZWXH7sOJgbqIA--MJwxobSp76Q>
 application/x-mpegURL
1496754871.194 15 192.168.0.100 TCP_MISS/200 16636 GET
http://qthttp.apple.com.edgesuite.net/1010qwoeiuryfg/0150_vod.m3u8
<http://www.google.com/url?q=http%3A%2F%2Fqthttp.apple.com.edgesuite.net%2F1010qwoeiuryfg%2F0150_vod.m3u8&sa=D&sntz=1&usg=AFQjCNHpiHy55EMeBIaMGhgEKRHanTrXxg>
-
HIER_DIRECT/95.101.182.201
<http://www.google.com/url?q=http%3A%2F%2F95.101.182.201&sa=D&sntz=1&usg=AFQjCNE9ZWXH7sOJgbqIA--MJwxobSp76Q>
 application/x-mpegURL
1496754871.715 4 192.168.0.100 TCP_MISS/200 16636 GET
http://qthttp.apple.com.edgesuite.net/1010qwoeiuryfg/0150_vod.m3u8
<http://www.google.com/url?q=http%3A%2F%2Fqthttp.apple.com.edgesuite.net%2F1010qwoeiuryfg%2F0150_vod.m3u8&sa=D&sntz=1&usg=AFQjCNHpiHy55EMeBIaMGhgEKRHanTrXxg>
-
HIER_DIRECT/95.101.182.201
<http://www.google.com/url?q=http%3A%2F%2F95.101.182.201&sa=D&sntz=1&usg=AFQjCNE9ZWXH7sOJgbqIA--MJwxobSp76Q>
 application/x-mpegURL

If I understand well, TCP_MISS/200 shows the content is not cached by
Squid. Could you please help me to see if anything I did wrong to make
.m3u8 not cached by Squid?

Date: Tue, 6 Jun 2017 14:08:11 +1200
From: Amos Jeffries <squid3 at treenet.co.nz>
To: squid-users at lists.squid-cache.org
Subject: Re: [squid-users] Squid issue of caching the m3u8 file
Message-ID: <5af5e6e9-4880-f58e-106b-9f11dd0a4a7b at treenet.co.nz>
Content-Type: text/plain; charset=utf-8; format=flowed

On 05/06/17 23:54, LIU Yaning wrote:
> Dear All,
>
> I would like to cache the .m3u8 file to be able to provide offline
> caching service by Squid. The played HLS video streaming is the link
> as below:
> http://qthttp.apple.com.edgesuite.net/1010qwoeiuryfg/0150_vod.m3u8
> <http://qthttp.apple.com.edgesuite.net/1010qwoeiuryfg/0150_vod.m3u8>
> However, the .m3u8 file is not be cached probably because it is
> mentioned as a no-cache, no-store, max-age=0 in the "Cache-Control" in
> the HTTP header.
Nope. Only the CC:no-store is preventing caching. The other headers
simply put boundaries on what is to be done with the content in the
cache. In particular the "no-cache", max-age=0 and Expires values mean
it has to be revalidated (REFRESH in your access.log) before any future
uses - probably because of that Set-Cookie needing to be changed for
different end-users.
>
> HTTP/1.1 200 OK
>
> Server: Apache
>
> ETag: "1d7168b4f49e75f76f3182f24bf075f6:1299516751"
>
> Last-Modified: Mon, 07 Mar 2011 16:52:31 GMT
>
> Expires: Fri, 02 Jun 2017 14:26:52 GMT
>
> Cache-Control: max-age=0, no-cache, no-store
>
> Pragma: no-cache
>
> Date: Fri, 02 Jun 2017 14:26:52 GMT
>
> Content-Length: 16046
>
> Set-Cookie: AKID=77F9F1316ECCE780566608C5E514DE0A;expires=Fri, 26 Aug
> 2016 00:01:00 GMT; path=/; domain=qthttp.apple.com.edgesuite.net
> <http://qthttp.apple.com.edgesuite.net/>
>
> Content-Type: application/x-mpegURL
>
> Access-Control-Allow-Origin: *
>
> I added a new rule for .m3u8 file in squid.conf, however, it is still
> not working.
>
> refresh_pattern -i \.(ts|m3u8)$ 120 90% 1000 override-expire
> override-lastmod ignore-no-cache ignore-no-store
>
> Does anyone know how to allow Squid caching the .m3u8 file? Thanks a
> lot in advance.
What makes you think it is not caching? The ignore-no-store alone should
be sufficient to allow current Squid versions to cache that object. You
could perhapse add "store-stale" option on that config line. Which
should make Squid cache object containing an Expires header with current
or past values. refresh_pattern settings do not affect that cacheable vs
non-cacheable decision. The no-cache header tells Squid the object needs
revalidating before every use. However, be aware the tool at redbot.org
tells me that this URL is badly broken in how it is using the ETag and
Vary headers - in a way which can break the revalidation when these
things are cached. Some of your clients may see very broken behaviour
accessing this object unless you follow the no-store requirement or the
server stops its broken ETag behaviour. PS. if you are using a Squid
version much older than 3.5.24 I recommend an upgrade. With an urgency
increasing the older your Squid is. Amos

-- 
Best Regards,
--
Yaning.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20170606/5cf5752a/attachment-0001.html>