[squid-users] external helper development

[Eliezer Croitoru]

Hey Andre,

Every language has a "cost" for it's qualities.
For example, Golang is a very nice language that offers a relatively simple way for concurrency support and cross hardware compilation/compatibility.
One cost in Golang is that the binary is in the size of an OS/Kernel.
In python you must write everything in a specific position and indentation and threading is not simple to implement for a novice.
However when you see what was written in Python you can see that most of OpenStack api's and systems are written in.. python and it means something.
I like very much ruby but it doesn't support threading by nature but supports "concurrency".
Squid doesn't implement threading but implements "concurrency".

Don't touch PHP as a helper!!! (+1 to Alex)

Also take into account that Redis or Memcached is less preferred in many cases if the library doesn't re-use the existing connection for multiple queries.
Squid also implements caching for helpers answers so it's possible to implement the helper and ACL's in such a way that squid caching will
help you to lower the access to the external API and or redis/memcahced/DB.
I also have good experience with some libraries which implements cache that I have used inside a helper with a limited size for "level 1" cache.
It's possible that if you will implement both the helper and server side of the solution like ufdbguard you would be able to optimize the system
to take very high load.

I hope the above will help you.
Eliezer

----
Eliezer Croitoru
NgTech, Tech Support
Mobile: +972-5-28704261
Email: ngtech1ltd at gmail.com

-----Original Message-----
From: squid-users <squid-users-bounces at lists.squid-cache.org> On Behalf Of André Bolinhas
Sent: Wednesday, February 2, 2022 00:09
To: 'Alex Rousskov' <rousskov at measurement-factory.com>; squid-users at lists.squid-cache.org
Subject: Re: [squid-users] external helper development

Hi
Thanks for the reply.
I will take a look on Rust as you recommend.
Also, between Python and Go and is the best for multithreading and concurrency?
Rust supports multithreading and concurrency?
Best regards

-----Mensagem original-----
De: squid-users <squid-users-bounces at lists.squid-cache.org> Em Nome De Alex Rousskov
Enviada: 1 de fevereiro de 2022 22:01
Para: squid-users at lists.squid-cache.org
Assunto: Re: [squid-users] external helper development

On 2/1/22 16:47, André Bolinhas wrote:
> Hi
> 
> I’m building an external helper to get the categorization of an 
> website, I know how to build it, but I need you option about the best 
> language for the job in terms of performance, bottlenecks, I/O blocking..
> 
> The helper will work like this.
> 
> 1º  will check the hot memory for faster response (memcache or redis)
> 
> 2º If the result not exist in hot memory then will check an external 
> api to fetch the categorie and saved it in hot memory.
> 
> In what language do you recommend develop such helper? PHP, Python, Go..

If this helper is for long-term production use, and you are willing to learn new things, then use Rust[1]. Otherwise, use whatever language you are the most comfortable with already (except PHP), especially if that language has good libraries/wrappers for the external APIs you will need to use.

Alex.
[1] https://www.rust-lang.org/
_______________________________________________
squid-users mailing list
squid-users at lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users

_______________________________________________
squid-users mailing list
squid-users at lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users