[squid-users] [SQUID] Some Web Page never complete download

[ngtech1ltd at gmail.com]

Great to hear all the details.
My general approach is to not intercept if possible.
In my Setup I am using squid only as an assisting software.
I wrote many tools to give squid features which are similar to other products like fortigate and checkpoint.
All the products in the market are using very basic inspections else then PaloAlto.
However, PaloAlto is an overkill for a tiny setup.
I like ufdbguard but I believe that there are other options which can be written in a much simpler way for my needs.
Ufdbguard is a great piece of software but I suggest you to use a DNSBL helper with a caching proxy for public DNS filtering services like cloudflare and others.
This way you don't need to maintain a local copy of the lists else then your custom to your choice.
I have used a set of unbound dns caching containers and a set of external helpers that are querying these caching dns services.
The overall effect is pretty good and the bandwidth worth it compared to the investment in maintaining lists.
Also you can use adguard as a container with specific lists compared to ufdbguard and a helper that runs queries against it.
The overall requirements are a bit higher but you can get clearer picture with statistics.

It would be very nice to try and sit on such a setup in a zoom meeting if it's of any interest for you to show me yours and I will show you mine.

Yours,
Eliezer

-----Original Message-----
From: slagauterie at hotmail.com <slagauterie at hotmail.com> 
Sent: Saturday, November 16, 2024 8:08 PM
To: ngtech1ltd at gmail.com; squid-users at lists.squid-cache.org
Subject: Re: [squid-users] [SQUID] Some Web Page never complete download

Hello Eliezer,

I use it as a standard forward proxy. I use a proxy.pac file deployed via DHCP and DNS WPAD entry. This makes it works easily with browsers.

10 years ago, I was doing both, but Squid was running on a DMZ server.
It was also acting as a firewall and other things.

Now, it is only forward. The main purpose is to prevent kids to access "wrong site" and remove ads.

Regarding C-ICAP, only squidclamav, connected to a dedicated Clamav container for virus checking.
Few years ago, ClamAV was in the same container, but for me it breaks a little bit the "One service" approach of conteiners.
I plan to also use DNSBL. But before, I need to fully understand how it can help or what it can improve.

Finally, I use ufdbguard instead of squidguard because it is still and well developed. It is also faster. Its main purpose is for URL checking. With it I mainly avoid:
- Trackers
- Ads
- Adults
- Violence, aggressive, weapon, etc.
- Warez

The next step will probably be to move ufdbguard out, in its dedicated container (One service approach).

At the begining, compiling was the main difficulty to produce Docker images. Then, I learn how to do multi-level image. So I can compile in one image, and use the result for other images, keeping the final image as small as possible, without all developper packages.

The main reasons why I currently don't use Squid to intercept are:
- My Docker server is a NAS. So not as powerfull as a real server. I tend to limit traffic.
- With the bump process and all the TLS approach, too many thing do not work out of the box for smartphone and their applications... Most of them are doing Certificate Pinning for example, which is broken by "standard" bumping and certificate mimic.

Note: I have not yet looked at all C-ICAP modules available. Feel free to recommend some.

Regards,
Slag

Le samedi 16 novembre 2024 à 17:55 +0200, ngtech1ltd at gmail.com a écrit :
> Hey Slag,
> 
> I want to understand the setup a bit more then what's written already.
> The Squid instance you are using, is it a simple forward proxy or an 
> interception one?
> The C-ICAP is used with SquidClamAV? Are there any other C-ICAP 
> functions you are using?
> What are you using UfdbGuard for? 
> 
> Thanks,
> Eliezer
> 
> -----Original Message-----
> From: squid-users <squid-users-bounces at lists.squid-cache.org> On 
> Behalf Of slagauterie at hotmail.com
> Sent: Thursday, November 14, 2024 8:15 PM
> To: squid-users at lists.squid-cache.org
> Subject: [squid-users] [SQUID] Some Web Page never complete download
> 
> Hello,
> 
> This is my first mail to this list, my apologies if things are not 
> correct.
> 
> I am using squid at home for more than 5 years. Recently I have 
> updated my version from version 6.0 (early age) to the latest version 
> 6.12.
> 
> I am compiling my own binaries and run it in a docker container.
> 
> Unfortunately, I am facing an issue with all Squid version above 6.8 
> (included).
> 
> I have some web sites like www.google.com for which my browser never 
> complete the download of the page. There is a pending request to URIs 
> like https://www.google.com/xjs/_/js/k=xjs.s.... and after long time
> (timeout) Squid logs a TCP_MISS_ABORTED/200.
> 
> I have downgraded till version 6.7, and it works a older version, even 
> if the same kind of request ends by a NONE_NONE_ABORTED/000. The 
> request does not stay pending.
> 
> I can provide more information if it can help to investigate.
> 
> Configuration:
> - Squid Version >= 6.8 (Compiled, with bumping configuration and 
> certificate mimic)
> - C-ICAP Version 0.6.3 (Compiled)
> - SquidClamAV Version 7.3 (Compiled)
> - UfdbGuard Version 1.35.8 (Compiled)
> 
> Thank you for your help.
> 
> Regards,
> Slag
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> https://lists.squid-cache.org/listinfo/squid-users
>