[squid-users] Basic example for store.log analyzer

[Eliezer Croitoru]

Since there is a bug in the store.log this a fixed version of the tool:
http://paste.ngtech.co.il/pll97qera

The bug is that the cache_mem should not be reported on each memory 
store object release inside the store.log since the store.log is for 
cache_dir only.
The difference between a cache_dir and a memory object release is the 
file number(the 4 argument in a log line) which is "FFFFFFFF" for memory 
object and lower for a regular cache_dir such as ufs.

Eliezer

* Thanks to Amos and Alex for their help to understand the issue.

On 01/10/2015 16:21, Eliezer Croitoru wrote:
> I already had a plan to write something like that in the past and I had
> some time so I wrote this store.log tool:
> http://paste.ngtech.co.il/pr3kbbf4q
>
> The tool is written in ruby and what it does is "estimating" what is in
> the cache_dir now based on reading the store.log.
>
> Since I have not spent too much time on understanding the store.log but
> I had a basic idea of whats in it that seems to give some results for now.
>
> The tool gets only one argument and it's the location of squid store.log
> and reads it like the store "journal" which takes view from nothing to
> what should exit now.
> Each line in the store.log represents one operation and it is expected
> to be logger in the order of execution.
> Due to this expectation we can predict that if a certain file was
> written to the disk(using SWAPOUT) and until the end of the log(which
> should represent now) it was not reported to be removed(RELEASE) from
> the cache it is still there but there is no guarantee that it will be
> used as a cache HIT.
>
> The tool needs more functionality to be more accurate and to display the
> estimated cache_dir size.
> For now running the script piping it with "wc -l"(reduce 1 line) will
> give you the result of how many objects you have in all your cache_dir
> on the server from the start time of the store.log.
>
> Any suggestions and requests regarding the tool are welcome.
>
> Eliezer
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users