<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@MS Gothic";
panose-1:2 11 6 9 7 2 5 8 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal>Did you mean 1 CPU with couple cores?<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>For squid it most of the time takes time to load these lists into ram.<o:p></o:p></p><p class=MsoNormal>It’s not wrong to do so since in many cases it’s the right thing to do.<o:p></o:p></p><p class=MsoNormal>In case these lists are stale for at-least a day I assume it should be fine.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>However there are tools like ufdbguard and others which are very good in the sense of<br>memory footprint and fast URLs lookup.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>How do you add these URLS, I am not mistaken about URLs and not domains right?<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Eliezer<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>----<o:p></o:p></p><p class=MsoNormal>Eliezer Croitoru<o:p></o:p></p><p class=MsoNormal>Tech Support<o:p></o:p></p><p class=MsoNormal>Mobile: +972-5-28704261<o:p></o:p></p><p class=MsoNormal>Email: <a href="mailto:ngtech1ltd@gmail.com">ngtech1ltd@gmail.com</a><o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b>From:</b> m k <tamurin0525@gmail.com> <br><b>Sent:</b> Thursday, August 6, 2020 8:29 AM<br><b>To:</b> Eliezer Croitor <ngtech1ltd@gmail.com><br><b>Cc:</b> squid-users@lists.squid-cache.org<br><b>Subject:</b> Re: [squid-users] I would like to know performance sizing aspects.<o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal>Eliezer,<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Squid's default setting is 1 core CPU, 16GB mem.<o:p></o:p></p></div><div><p class=MsoNormal>How many URLs(Blacklist) will degrade Squid's performance?<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Also, SSL-Bump.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Thank you,<o:p></o:p></p></div><div><p class=MsoNormal>kitamura<o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal>2020<span style='font-family:"MS Gothic"'>年</span>8<span style='font-family:"MS Gothic"'>月</span>6<span style='font-family:"MS Gothic"'>日</span>(<span style='font-family:"MS Gothic"'>木</span>) 13:38 Eliezer Croitor <<a href="mailto:ngtech1ltd@gmail.com">ngtech1ltd@gmail.com</a>>:<o:p></o:p></p></div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in'><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Kitamura,<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>About the tens of thousands of URLs, Have you considered using a Blacklisting utility, it might lower the memory footprint.<o:p></o:p></p></div></div><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Eliezer<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>----<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Eliezer Croitoru<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Tech Support<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Mobile: +972-5-28704261<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Email: <a href="mailto:ngtech1ltd@gmail.com" target="_blank">ngtech1ltd@gmail.com</a><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b>From:</b> squid-users <<a href="mailto:squid-users-bounces@lists.squid-cache.org" target="_blank">squid-users-bounces@lists.squid-cache.org</a>> <b>On Behalf Of </b>m k<br><b>Sent:</b> Thursday, August 6, 2020 7:25 AM<br><b>To:</b> Amos Jeffries <<a href="mailto:squid3@treenet.co.nz" target="_blank">squid3@treenet.co.nz</a>><br><b>Cc:</b> <a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a><br><b>Subject:</b> Re: [squid-users] I would like to know performance sizing aspects.<o:p></o:p></p></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Amos,<o:p></o:p></p></div></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Thank you for your reply.<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>It was very helpful.<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>> That number was gained before HTTPS became so popular. So YMMV depending<br>> on how many CONNECT tunnels you have to deal with. That HTTPS traffic can possibly be decrypted <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>> and cached but performance trade-offs are quite large.<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Squid uses SSL-Bump.<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>I'm very worried about the internet slowing down due to https decording. and I'm also worried about the internet slowing down due to using Blacklist.<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>I load tens of thousands of URL(black list file) every time I set up ACL.<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>How many requests does SSL-Bump in one second?<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Thank you,<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>kitamura<o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>2020<span style='font-family:"MS Gothic"'>年</span>8<span style='font-family:"MS Gothic"'>月</span>5<span style='font-family:"MS Gothic"'>日</span>(<span style='font-family:"MS Gothic"'>水</span>) 10:32 Amos Jeffries <<a href="mailto:squid3@treenet.co.nz" target="_blank">squid3@treenet.co.nz</a>>:<o:p></o:p></p></div></div></div></div><div><div><div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt'><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>On 5/08/20 11:28 am, m k wrote:<br>>> We are considering to use Squid for our proxy, and would like to know<br>>> performance sizing aspects.<br>>><br>>> Current web access request averages per 1 hour are as followings <br>>> Clients<span style='font-family:"MS Gothic"'>：</span>30,000<span style='font-family:"MS Gothic"'>、</span><br>>> Page Views:141,741/hour<br>>> *Requests:4,893,106<br>>><br><br>Okay. Requests and client count are the important numbers there.<br><br>The ~1359 req/sec is well within a default Squid capabilities, which can<br>extend up to around 10k req/sec before needing careful tuning.<br><br>That number was gained before HTTPS became so popular. So YMMV depending<br>on how many CONNECT tunnels you have to deal with. That HTTPS traffic<br>can possibly be decrypted and cached but performance trade-offs are<br>quite large.<o:p></o:p></p></blockquote><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt'><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><br>>> We will install Squid on CentOS 8.1. Please kindly share your<br>>> thoughts / advices<br><br>Whatever OS you are most comfortable with administering. Be aware that<br>CentOS official Squid packages are very slow to update - Apparently they<br>still have only v4.4 (8 months old) despite a 8.2 point release only a<br>few weeks ago.<br><br>So you may need to be building your own from sources and/or using other<br>semi-official packagers such as the ones from Eliezer at NGTech when he<br>gets around to CentOS 8 packages.<br> <<a href="https://wiki.squid-cache.org/KnowledgeBase/CentOS" target="_blank">https://wiki.squid-cache.org/KnowledgeBase/CentOS</a>><br><br><br>FYI; If you find yourself having to use SSL-Bump, then we highly<br>recommended to follow the latest Squid releases with fairly frequent<br>updates (at minimum a few times per year - worst case monthly). If you<br>like CentOS you may find Fedora more suitable to track the security<br>environment volatility and update churn.<br><br><br>>> Is there sizing methodology and tools?<br><br>There are a couple of methodologies, depending on what aspect you are<br>tuning towards - and one for identifying the limitation points to begin<br>a tuning process tuning.<br><br>The info you gave above is the beginning. Checking to see if your<br>traffic rate is reasonably within capability of a single Squid instance.<br><br>Yours is reasonable, so next step is to get Squid running and see where<br>the trouble points (if any) are.<br><br> For more see <<a href="https://wiki.squid-cache.org/SquidFaq/" target="_blank">https://wiki.squid-cache.org/SquidFaq/</a>><br><br><br><br>>> How much resources are generally recommended for our environment?<br>>><span style='font-family:"MS Gothic"'> </span> CPU:<span style='font-family:"MS Gothic"'> </span> Memory:<span style='font-family:"MS Gothic"'> </span> Disk space : Other factors to be considered if any:<br>>> Do you have a generally recommended performance testing tools? Any<br>>> suggested guidelines?<br>>><br><br><br> CPU - squid is still mostly single-process. So prioritize faster GHz<br>rates over core number. Multi-core can help of course, but not as much<br>as cycle speeds do. Hyper-threading is useless for Squid.<br><br> Memory - Squid will use as much as you can give it. Let your budget<br>govern this.<br><br> Disk - Squid will happily run with no disk - or lots of large ones.<br><br> - Avoid RAID. Squid *will* shorten disk lifetimes with its unusually<br>high write I/O pattern. How much shorter varies by disk type (HDD vs<br>SSD). So you may find it better to plan budget towards maintenance costs<br>of replacing disks in future rather than buying multiple up-front for<br>RAID use.<br> see <<a href="https://wiki.squid-cache.org/SquidFaq/RAID" target="_blank">https://wiki.squid-cache.org/SquidFaq/RAID</a>> for details.<br><br> - Up to a few hundred GB per cache_dir can be good for large caches.<br>Going up to TB is not (yet) worth the disk cost as Squid has a per-cache<br>limit on stored objects.<br><br> - Disk caches can be re-tuned, added, moved, removed, and/or extended<br>at any time and will depend on the profile of object sizes your proxy<br>handles - which itself likely changes over time. So general let your<br>budget decide the initial disks and work from there.<br><br><br><br>Load Testing - the tools us dev use to review performance are listed at<br>the bottom of the profiling FAQ page. These are best for testing the<br>theoretical limits of a particular installation - real traffic tends to<br>be somewhat lower. So I personally prefer taking stats from the running<br>proxy on real traffic and seeing what I can observe from those.<br><br><br>HTH<br>Amos<br>_______________________________________________<br>squid-users mailing list<br><a href="mailto:squid-users@lists.squid-cache.org" target="_blank">squid-users@lists.squid-cache.org</a><br><a href="http://lists.squid-cache.org/listinfo/squid-users" target="_blank">http://lists.squid-cache.org/listinfo/squid-users</a><o:p></o:p></p></blockquote></div></div></div></div></div></blockquote></div></div></div></body></html>