[squid-users] Peer2Peer Url categorizing, black\white lists, can it work?

[Eliezer Croitoru]

I have it on my plate for quite some time and I was wondering about the
options and interest in the subject.

Intro:
Currently most free blacklists are distributed in the old fashion way of a
tar or other file.
There are benefits to these but I have not seen an option to be able to
"help" each other.
For example many proxy servers "knows" about a domain that other do not.
So even if the site exists and know in one side of the planet it's not in
another.
If it could be categorized or white\black listed in one side of the planet
why we cannot help each other?
Many admins adds sites to their DB and list but not many share them
publically.

The idea:
As an example Google and Mozilla services advertise malware infected sites
using their browser.
Many filtering solutions uses their clients logs to inspect and enhance
their lists.
There are many distributed key+value DB systems such as etcd and many others
DHT based.
I believe that somehow a url categorizing and black\white lists can be
advertised in a similar way.
The only limit is the "bootstap" or the "routers" of such a network.
Since such a service should only apply to KEYS and values which today should
not exceed 1MB I believe it would be pretty simple to create networks based
on that.
Once a network category or scheme can be defined it would be pretty simple
to "match" or "connect" between the relevant nodes.

Currently I am looking at the different options for the backend DB,
permissions and hierarchy which should give an admin a nice start point.
Such "online network" can be up and running pretty fast and it can enhance
the regular categories and lists to be more up-to-date.
Else then the actual categorizing and listing I believe that it would be
possible to share and generate a list of public domains which are known
compared to the current state which many parts of the web is "unknown".

If you wish to participate in any of the above ideas please contact me here
or privately.

Eliezer

----
Eliezer Croitoru <http://ngtech.co.il/lmgtfy/> 
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer at ngtech.co.il
 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: winmail.dat
Type: application/ms-tnef
Size: 64421 bytes
Desc: not available
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20160726/8a1a49a0/attachment-0001.bin>