[squid-users] Squid test-suite / benchmarks

[Alex Rousskov]

On 06/17/2018 02:28 AM, Stoica Bogdan Alexandru wrote:
> Thank you all for your suggestions. Polygraph is a good benchmark, but
> unfortunately it has a strict terms & conditions when it comes to
> publishing results and we plan to make the measurements part of a
> research paper.

Polygraph license (Apache v2) does not restrict or regulate publication
of the results.

Alex.

  

> *From:*Coenraad Loubser [mailto:coenraad at wish.org.za]
> *Sent:* Thursday, June 7, 2018 11:26 AM
> *To:* Stoica Bogdan Alexandru <bogdan.stoica at epfl.ch>
> *Cc:* squid-users at lists.squid-cache.org; Alex Rousskov
> <rousskov at measurement-factory.com>
> *Subject:* Re: [squid-users] Squid test-suite / benchmarks
> 
>  
> 
> My first port of call would be apachebench with and without your
> proxies. A web search for "squid apachebench" might yield some leads to
> people who have done this. (I'm sure apachebench is well tested.)
> 
> Eg. the third hit on Google:
> https://2bits.com/articles/using-apachebench-benchmarking-logged-users-automated-approach.html
> 
>  
> 
> On Thu, Jun 7, 2018 at 7:20 PM, Alex Rousskov
> <rousskov at measurement-factory.com
> <mailto:rousskov at measurement-factory.com>> wrote:
> 
>     On 06/07/2018 04:17 AM, Stoica Bogdan Alexandru wrote:
> 
>     > We’re a small research team interested in benchmarking Squid for a
>     > research project.
> 
>     > Ideally, we would like to have good code coverage while doing so.
> 
>     > Are there any good benchmarks used for such purpose?
> 
>     Performance benchmarks usually focus on things other than code coverage.
>     It is very difficult to write a quality benchmark for a proxy, even
>     without code coverage as a goal!
> 
>     One the other hand, a decent proxy benchmark has enough knobs to tickle
>     most "interesting" code paths in Squid (or any other proxy). Web
>     Polygraph[1] (mentioned on this thread earlier) is a good example -- you
>     can trigger cache revalidation, simulate heavy tailed hit distributions
>     that stress disk caching, exercise the code that handles aborted
>     transactions, persistent connection races, etc., etc.
> 
> 
>     > Or, even better, is
>     > there a more comprehensive test suite apart from the one Squid
>     comes with?
> 
>     Squid does not come with a comprehensive test suite (yet) and the tests
>     distributed with Squid are not performance tests (a.k.a. "benchmarks").
>     If you are looking for functionality rather than performance testing,
>     then there is Co-Advisor[2]. Squid is tested with Co-Advisor, but those
>     tests have not been automated (yet).
> 
>       [1] http://www.web-polygraph.org/
>       [2] http://coad.measurement-factory.com/
> 
> 
>     HTH,
> 
>     Alex.
>     P.S. Disclaimer: The company I work for is responsible for both of the
>     test tools mentioned above.
> 
>     _______________________________________________
>     squid-users mailing list
>     squid-users at lists.squid-cache.org
>     <mailto:squid-users at lists.squid-cache.org>
>     http://lists.squid-cache.org/listinfo/squid-users
> 
> 
> 
> 
> -- 
> 
> 
> Coenraad Loubser
> 
> Wireless Internet Services & Hardware (Pty) Ltd.
> 210 Long Street, Cape Town, 8001, ZA
> 
> Office: +27 21 481 1824
> Skype: Coenraad_Loubser
> Email: coenraad at wish.org.za <mailto:coenraad at wish.org.za>
> Cell: +27 73 772 1223
> 
> -- Spending Money is like watering a plant.
>