[squid-users] CPU Load 100% after implementing SSL Bump ....

Mon May 23 08:18:16 UTC 2016

Hi Team,

Squid - Version 3.5.13

Please find the below Squid Cache Logs
2016/05/23 13:35:55 kid1| Error negotiating SSL connection on FD 138:
error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:35:55 kid1| Error negotiating SSL connection on FD 457:
error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:36:00 kid1| Error negotiating SSL connection on FD 33:
error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:36:01 kid1| Error negotiating SSL connection on FD 438:
error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:36:05 kid1| Error negotiating SSL connection on FD 555: (104)
Connection reset by peer
2016/05/23 13:36:06 kid1| Error negotiating SSL connection on FD 512:
error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:36:09 kid1| Error negotiating SSL connection on FD 618: (104)
Connection reset by peer
2016/05/23 13:36:15 kid1| Error negotiating SSL connection on FD 514:
error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:36:15 kid1| Error negotiating SSL connection on FD 206:
error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate unknown
(1/0)
2016/05/23 13:36:18 kid1| Error negotiating SSL connection on FD 627: (104)
Connection reset by peer
2016/05/23 13:36:18 kid1| Error negotiating SSL on FD 147:
error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
(1/0/0)
2016/05/23 13:36:19 kid1| Error negotiating SSL connection on FD 343:
error:14094416:SSL routines:SSL3_READ_BYTES:sslv3 alert certificate unknown
(1/0)
2016/05/23 13:36:24 kid1| Error negotiating SSL connection on FD 378: (104)
Connection reset by peer
2016/05/23 13:36:25 kid1| Error negotiating SSL connection on FD 491:
error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:36:28 kid1| ctx: enter level  0: '
http://afs.moatads.com/empty_flash?tracer='
2016/05/23 13:36:28 kid1| keepaliveAccounting: Impossible keep-alive header
from 'http://afs.moatads.com/empty_flash?tracer='
2016/05/23 13:36:34 kid1| ctx: exit level  0
2016/05/23 13:36:34 kid1| Error negotiating SSL connection on FD 257: (104)
Connection reset by peer
2016/05/23 13:36:34 kid1| Error negotiating SSL connection on FD 90:
error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:36:38 kid1| Error negotiating SSL on FD 125:
error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
(1/0/0)
2016/05/23 13:36:38 kid1| Error negotiating SSL connection on FD 577: (104)
Connection reset by peer
2016/05/23 13:36:38 kid1| Error negotiating SSL connection on FD 91: (104)
Connection reset by peer
2016/05/23 13:36:39 kid1| Error negotiating SSL connection on FD 220:
error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:36:43 kid1| Error negotiating SSL connection on FD 50: (104)
Connection reset by peer
2016/05/23 13:36:48 kid1| Error negotiating SSL connection on FD 579: (104)
Connection reset by peer
2016/05/23 13:36:48 kid1| Error negotiating SSL connection on FD 455: (104)
Connection reset by peer
2016/05/23 13:36:49 kid1| Error negotiating SSL connection on FD 414:
error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca (1/0)
2016/05/23 13:39:28 kid1| varyEvaluateMatch: Oops. Not a Vary match on
second attempt, '
http://cdn.sstatic.net/Sites/stackoverflow/all.css?v=fada5080e3ea'
'accept-encoding="gzip,%20deflate"'

----------------------------Cache log End
--------------------------------------

Do we need to update openssl? I got to know these from the forum previous
post ....
If we need to update the openssl then where can we find the updated version
of CA Certs ....

On Mon, May 23, 2016 at 12:52 PM, Amos Jeffries <squid3 at treenet.co.nz>
wrote:

> On 23/05/2016 6:27 p.m., Sagar Malve wrote:
> > Hi Team,
> >
> > System Config:
> >
> > Intel S2400SC2 Motherboard
> > Intel Xeon ES 2407 V2 CPU
> > RAM 32 GB
> >
>
> What Squid version?
>
> >
> > http_port 3127
> > http_port 3128 intercept
> > https_port 3129 intercept ssl-bump generate-host-certificates=on
> > dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_certs/squid.crt
> > key=/etc/squid/ssl_certs/squid.key options=NO_SSLv3
> > tls-dh=/etc/squid/dhparam.pem
> > sslproxy_capath /etc/ssl/certs
> >
> >
> > # FILTERING HTTPS
> > acl 1 dstdomain .fbcdn.net .akamaihd.net .fbsbx.com
> > #acl 2a dstdomain .mahadana.com .mql4.com .metaquotes.net
> > acl 2 url_regex -i ^https?:\/\/attachment\.fbsbx\.com\/.*\?(id=[0-9]*).*
> > acl 2 url_regex -i
> > \.fbsbx\.com\/.*\/(.*\.(unity3d|pak|zip|exe|dll|jpg|png|gif|swf)/)$
> > acl 2 url_regex -i ^https?:\/\/.*\.ytimg\.com(.*\.(webp|jpg|gif))
> > acl 2 url_regex -i ^https?:\/\/([^\.]*)\.yimg\.com\/(.*)
> > acl 2 url_regex -i ^https?:\/\/.*\.gstatic\.com\/images\?q=tbn\:(.*)
> > acl 2 url_regex -i
> >
> ^https?:\/\/.*\.reverbnation\.com\/.*\/(ec_stream_song|download_song_direct|stream_song)\/([0-9]*).*
> > acl 2 url_regex -i
> >
> ^https?:\/\/([a-z0-9.]*)(\.doubleclick\.net|\.quantserve\.com|.exoclick\.com|interclick.\com|\.googlesyndication\.com|\.auditude\.com|.visiblemeasures\.com|yieldmanager|cpxinteractive)(.*)
> > acl 2 url_regex -i ^https?:\/\/(.*?)\/(ads)\?(.*?)
> > acl 2 url_regex -i ^https?:\/\/.*steampowered\.com\/.*\/([0-9]+\/(.*))
> > acl 3 url_regex -i
> > ^https?:\/\/(.*?)\/speedtest\/.*\.(jpg|txt|png|gif|swf)\?.*
> > acl 3 url_regex -i speedtest\/.*\.(jpg|txt|png|gif|swf)\?.*
> > acl 4 url_regex -i reverbnation.*audio_player.*ec_stream_song.*$
> > acl 5 url_regex -i utm.gif.*
> > acl 6 url_regex -i
> c.android.clients.google.com.market.GetBinary.GetBinary.*
> > acl 7 url_regex -i youtube.*(ptracking|stream_204|player_204|gen_204).*$
> > acl 7 url_regex -i
> > \.c\.(youtube|google)\.com\/(get_video|videoplayback|videoplay).*$
> > acl 7 url_regex -i (youtube|google).*\/videoplayback\?.*
> > acl 8 http_status 302
> > acl getmethod method GET
> >
>
> Using .* on the beginning or end of a regex does nothing but cause more
> CPU workload for Squid.
>
> If you put it inside (.*), or with an anchor ^.* or .*$ just makes the
> CPU usage worse.
>
> What http_access rules are using those?
>
> >
> > ssl_bump splice localhost
> > acl 9 at_step SslBump1
> > acl 10 at_step SslBump2
> > acl 11 at_step SslBump3
> > ssl_bump peek 9 all
> > ssl_bump bump 10 all
> > ssl_bump bump 11 all
>
> Step3 of bumping process will never happen. You told Squid to begin
> decryption at step2.
>
> Have you disabled "via"?
>
>
> >
> >
> ----------------------------------------------------------------------------------------------
> >
> > Is there any way where it can Cache SSL Certificate for all HTTPS Traffic
> > ....
> > Because SSL Cert & Squid process were using 99% of CPU Load ....
>
> Er, what do you think caching does exactly?
>
> Caching HTTPS will have no effect on your described CPU problem. Might
> make it worse even.
>
>
> Between them?
>
> How much is each process using?
>
> How may concurrent connections are being handled by Squid to get that
> loading ?
>
>
> Check whether Squid is finished loading its cache_dir indexes, or if any
> of them are undergoing a "DIRTY" rebuild. That can use a lot of CPU
> while its happening and caching cannot be fully operational until its
> finished either.
>
>
> >
> > We have approx 200 users ....
> >
> > I have set the open file limit to 100000
>
> FYI: SSL-Bump with your configuration will use 3 FD for each client
> inbound HTTPS request. That 100K limit will restrict your users to 150
> concurrent connections each.
> A browser using Happy eyeballs will open 16 connections to each domain.
> Average web page on the most popular sites involve around 100 objects
> spread over 10+ domains.
>   => ~160 FD needed to load an average page.
>
> I'd double that limit, if you expect this proxy to have much traffic.
>
> >
> > Could you please let us know if there is any way to Cache the HTTPS
> Request
> > in Squid .....
> >
>
> You are already SSL-Bumping traffic. That removes the 'S' from HTTPS.
> Leaving Squid with regular HTTP messages, which already are cached if it
> can.
>
> Amos
>
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20160523/692a3b1a/attachment-0001.html>