[squid-users] Rock datastore, CFLAGS and a crash that (may be) known

[Jester Purtteman]

Thanks Eliezer.  You are right, esxi timing is all sorts of wonky, and as few a cores with as much speed dedicated as possible seems to be the best answer I have found.  I might actually rebuild this sucker on a raw linux box and ditch the esxi host shindig, just adds bloat when the only virtual machine you're running is squid anyway.  There was a case made for multiple systems on this thing, a while ago, but that case is getting weak and we have other machines to do that now.

> -----Original Message-----
> From: Eliezer Croitoru [mailto:eliezer at ngtech.co.il]
> Sent: Wednesday, February 17, 2016 8:44 AM
> To: Jester Purtteman <jester at optimera.us>; squid-users at lists.squid-
> cache.org
> Subject: Re: [squid-users] Rock datastore, CFLAGS and a crash that (may be)
> known
> 
> So after reading the whole thread from top to bottom:
> Since it's an ESXi version 6.0 with an Ubuntu 14.04 guest I would choose
> another approach!
> You do not use any SSL related settings and it's a simple proxy(from the squid
> -v output) so I would ask "did you tried to rebuild any deb file?"
> There might be some benefits from a special flag or more but from my
> experience with such VMs it would be very little in most cases.(I do not have
> tons of experience...) I had experience with a bunch of Gentoo machines
> running all sort of web and Internet services on them for years. The claim for
> self compiling was that it is much efficient then pre-built binaries. The fact is
> that real hardware was faster\better then VMs (ESXi) either with pre-built or
> self-compiled binaries. So when moved from HW to VMs there were lots of
> things to test and confirm. Things like high CPU spikes or high DISK IO activity
> spikes which was pretty weird compared to the real HW.
> Currently I have seen that VMs given enough vCPU, RAM and doing some
> fine tuning for balancing between VMs(..not throwing 24 vCPU per guest on
> a 24 cores host) and hosts gave for *these specific hosts* better
> performance then custom-compiling and flagging.
> 
> I have not built a Debian\Ubuntu deb package for a very long time but I had a
> plan to do so.
> Maybe I will do it one day.
> 
> All The Bests,
> Eliezer Croitoru
> 
> On 17/02/2016 15:36, Jester Purtteman wrote:
> > Dear Eliezer, Amos and Marcus,
> >
> > Thank you, and sorry for the late reply, day jobs are a menace to
> > productivity:)
> >
> > So, in order of responses:  Eliezer:
> >> >Before digging into the details of the issue, can you supply the OS
> details?
> >> >What OS are you using? What distribution?
> >> >32 or 64 bit?
> >> >can you also add the output of "squid -v" for both 3.5.14 and 3.5.13 ?
> > I am running Ubuntu 14.04.2 updated to the latest apt-get binaries, 64-bit
> version, 4 processors, 24-gb of "ram" allocated under the VM.  This is all on a
> vmware ESXi 6.0 host, so I recognize that compiler flags are probably a bit like
> throwing water balloons at Jaws from a performance stand point.  The
> counter point is, with performance as bad as a VM, you need all the help you
> can get.  As much as anything, it was a curiousity.
> >
> > Squid -v for a working configuration is as follows:
> >
> > Squid Cache: Version 3.5.14
> > Service Name: squid
> > configure options:  '--with-pthreads' '--prefix=/usr'
> > '--localstatedir=/var' '--libexecdir=/usr/lib/squid' '--srcdir=.'
> > '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid'
> > '--with-default-user=proxy' '--with-logdir=/var/log'
> > '--with-pidfile=/var/run/squid.pid' '--enable-linux-netfilter'
> > '--enable-cache-digests' '--enable-storeio=ufs,aufs,diskd,rock'
> > '--enable-async-io=30' '--enable-http-violations' '--enable-zph-qos'
> > '--with-netfilter-conntrack' '--with-filedescriptors=65536'
> > '--with-large-files' --enable-ltdl-convenience
> >
> > I cut and pasted the configuration string I'd used with 3.5.13, added "--with-
> pthreads", and had no problems.  Here is the working 3.5.13 -v output:
> >
> > Squid Cache: Version 3.5.13
> > Service Name: squid
> > configure options:  '--prefix=/usr' '--localstatedir=/var'
> > '--libexecdir=/usr/lib/squid' '--srcdir=.'
> > '--datadir=/usr/share/squid' '--sysconfdir=/etc/squid'
> > '--with-default-user=proxy' '--with-logdir=/var/log'
> > '--with-pidfile=/var/run/squid.pid' '--enable-linux-netfilter'
> > '--enable-cache-digests' '--enable-storeio=ufs,aufs,diskd,rock'
> > '--enable-async-io=30' '--enable-http-violations' '--enable-zph-qos'
> > '--with-netfilter-conntrack' '--with-filedescriptors=65536'
> > '--with-large-files' --enable-ltdl-convenience