[squid-users] 3.3.x -> 3.4.x: huge performance regression

[Alan]

I had the same problem when I tried to move from 3.3 to 3.4.  Because
of this, I had to go back to 3.3.

I don't remember the CPU being stuck at 100%, but it was certainly
higher, and the Internet browsing experience was slower.

I don't use the NTLM helper, but I do use the Kerberos one, and also a
couple of custom external acls.

Alan


On Fri, Oct 24, 2014 at 9:49 PM, Eliezer Croitoru <eliezer at ngtech.co.il> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hey Eugene,
>
> I will try to clear things out.
> Any helper is an external software but each and every one of them
> answers to a specific logic.
> external_acl url_rewrite and store_id helpers works in one way while
> authentication works in another.
>
> To make the story short external_acl url_rewrite and store_id helpers
> acquire a single line from squid via STDIN and responses to his own
> STDOUT towards squid STDIN.
> an example would be:
> 1 http://www.yahoo.com "more things such as requester and others"
>
> The helper responses to this with a line such as:
> 1 ERR
> or
> 1 OK "more stuff"
>
> The above example applies only to squid 3.4 while in squid 3.3 this is
> being used only on the external_acl interface.
>
> In auth helpers there are couple "stages" that applies on each requests.
> In external_acl and the similar interfaces the client do not need to
> send any details for the helper to analyze else then these that exists
> on each request.
> Auth helpers has a logic that denies with request for more details.
> NTLM specifically has overload on the whole process since it can have
> 3 stages against squid.
> Kerberous has the benefit of only 2 stages against squid:
> 1 - request -> denial
> 2 - request + credentials -> denial or approval
>
> Comparing NTLM to external_acl helpers it is very different.
> But comparing Kerberous to external_acl it's almost similar.
>
> If for example NTML + Kerberous + external_acl + others suffers from
> the same issue while using only one of them at a time it will prove
> that there is a specific direction to the issue.
> But if Kerberous and NTLM differ it will prove that there are other
> things in hand.
>
> Another approach to test the issue is to write an ECAP or ICAP helper
> that will implement the whole authentication logic.
> This will also prove that the helpers code is broken in 3.4 compared
> to 3.3.
>
> There is also the comparison of basic authentication compared to NTLM
> + Keberous which is also important.
>
> The squid team needs a testing environment for the issue for quite
> some time.
>
> Now that Victor Sudakov has used Kerberous instead of NTLM we might be
> able to put some efforts and to pinpoint the issue.
>
> All The Bests,
> Eliezer
>
> On 10/24/2014 03:18 PM, Eugene M. Zheganin wrote:
>>> see http://bugs.squid-cache.org/show_bug.cgi?id=3997
>> There seems to be a large mix of terms, like "external helper" and
>> "external program". The ntlm_auth, which is external to squid and
>> part of the samba package, doesnt' change between version
>> uprgrades, so I don't see how it can be involved (and why replace
>> it with a fake helper ?). In the same time it's an authentication
>> helper. The squid_kerb_ldap, also known as
>> ext_kerberos_ldap_group_acl, is, indeed, an "external acl" helper,
>> but it has nothing to do with NTLM, as it doesn't use it. My
>> opinion - this pr is flooded with error messages of almost any
>> kind, and it's very hard to see the point.
>>
>> We need to understand what's happening to squid when it starts to
>> hit the 100% CPU usage, last time I saw this during the night
>> without any possible user traffic load.
>>
>> Eugene.
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1
>
> iQEcBAEBAgAGBQJUSkrVAAoJENxnfXtQ8ZQUp8UIAI2MUcnASF98RWQECoKFczS9
> E7SWOZMjQQLIy80NAId9MdiZ79XPNTF6a1VaGcdP4cAc78XgNqk6NdBF0eFIKSgy
> 31W48GGt8p0spJ0o8+NVijV9O/2pGcIW8M3psbnYG/fcgAzWq4DLz9Q/tFTtHsIa
> UzY56BMuSX1+0d2IeZSO2bXxEnOyiyAJKrQ96H/7tlg/9VV5mHOB7Py1b6G+O1YP
> nU/+vj4GppHYOYwPdqX9MwKfXwHzFPLHGXdafIJT1/Ci1ApEWW+137PMrvG1uvg3
> pzSF9amhAyBpAjIA1WRgoYb5gtUciK43K0EIXYkmZnmmuw3mEFOv4WWyDApFBgs=
> =TLyW
> -----END PGP SIGNATURE-----
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users