[squid-users] Squid for Windows Repeatedly Crashing
eliezer at ngtech.co.il
eliezer at ngtech.co.il
Wed Feb 27 22:28:13 UTC 2019
Forgot to mention that this simple proxy:
http://gogs.ngtech.co.il/elicro/golang-http-proxy
Is a simple forward proxy I wrote.
A binary packaged for any OS that GoLang supports including Windows 2k16 is there:
http://gogs.ngtech.co.il/NgTech-LTD/golang-http-proxy/src/master/golang-http-proxy.tar.xz
You will need some software to make it a service but these are easy to find.
If you need a recommendation for one I will try to find.
All The Bests,
Eliezer
----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer at ngtech.co.il
-----Original Message-----
From: squid-users <squid-users-bounces at lists.squid-cache.org> On Behalf Of eliezer at ngtech.co.il
Sent: Wednesday, February 27, 2019 22:44
To: 'Van Order, Drew (US - Hermitage)' <dvanorder at deloitte.com>
Cc: squid-users at lists.squid-cache.org
Subject: Re: [squid-users] Squid for Windows Repeatedly Crashing
So just to be on the clear.
You need a basic forward proxy that does CONNECT requests for any LAN client
to safe ports?
Do you have any other requirements then being Windows 2k16 compatbile and
the mentioned ACL's?
If you Insist on a Windows proxy there are couple I collected...
http://www1.ngtech.co.il/wpe/2016/05/02/proxy-per-internet-user-is-it-realis
tic/
I know that RedWood might be good for your needs to compare...:
http://ngtech.co.il/static/redwood/redwood-0.2.0.tar.xz
https://github.com/andybalholm/redwood
I wrote a tiny proxy the other day which should also work fine for you as
long as you have a working and properly configured firewall on the Server.
Let me know if something fit your needs.
If so you can try and test and maybe find the right culprit(ie windows or
linux).
Eliezer
----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer at ngtech.co.il
-----Original Message-----
From: Van Order, Drew (US - Hermitage) <dvanorder at deloitte.com>
Sent: Wednesday, February 27, 2019 17:55
To: eliezer at ngtech.co.il
Cc: squid-users at lists.squid-cache.org
Subject: RE: [squid-users] Squid for Windows Repeatedly Crashing
Business objective is to enable MSFT Azure MMA's (Microsoft Monitoring
Agents) blocked from the internet to send agent data to Azure Log Analytics
Simple proxy
No SSL bump
Squid config is attached
I tried disabling caching with Squid, found it crashed more frequently
Squid was chosen this is intended to be a stopgap solution, and it's free.
It's a battle to win over security in order to have tcp/443 opened
everywhere.
I'm not sure Squid is the problem, I have an identically configured Squid
that bypasses the F5 working beautifully, but it's only 50 clients (MMA's)
connecting, Each client takes roughly 5 connections. The clients are still
going through a firewall(s).
Our network folks say that neither the FW or F5 leading up to Squid report
congestion.
It's possible that Squid for Windows + F5 VIP are not intended to work
together, but it makes sense to just have one proxy IP address.
I'm getting ready to Skype with our F5 guy to compare what I'm seeing with
what he's seeing. Also trying to get how many clients are going through the
F5 to compare to my 'good' Squid
-----Original Message-----
From: eliezer at ngtech.co.il <eliezer at ngtech.co.il>
Sent: Wednesday, February 27, 2019 9:20 AM
To: Van Order, Drew (US - Hermitage) <dvanorder at deloitte.com>
Cc: squid-users at lists.squid-cache.org
Subject: [EXT] RE: [squid-users] Squid for Windows Repeatedly Crashing
The setup itself is not clear to me.
Is it a simple proxy?
With SSL bump?
Can you share or send me the squid configuration?
There might be another solution for your use case that you have yet to try.
Also if the purpose is not caching, why do you try to use squid?
There are lots of other proxies for windows out there? (just wondering what
and why have you choose Squid)
Thanks,
Eliezer
----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer at ngtech.co.il
-----Original Message-----
From: Van Order, Drew (US - Hermitage) <dvanorder at deloitte.com>
Sent: Wednesday, February 27, 2019 05:51
To: Eliezer Croitoru <eliezer at ngtech.co.il>; Rafael Akchurin
<rafael.akchurin at diladele.com>
Cc: 'Amos Jeffries' <squid3 at treenet.co.nz>;
squid-users at lists.squid-cache.org
Subject: RE: [squid-users] Squid for Windows Repeatedly Crashing
Hello folks, and thanks for keeping interest. Today I spent a bit of time
learning squidclient, and have determined that the server is not in any way
resource constrained. I've attached the output from mgr:info,
mgr:client_list, and mgr:filedescriptors in between crashes. Was wondering
if someone could explain Tout, which I presume is timeout. Of interest are
the ones set to 86400, which I presume is one day. That seems like a big
problem--but where is it coming from? I'm using the Cygwin Squid config
defaults.
There seems to be a lot of Reading next request going on before Squid
recycles. I wonder if the F5 VIP is dealing with congestion through the
firewall, which, in turn, is causing congestion on the pool output side, the
10.26.25.220 address. Our F5 guys have gone silent on me, I have been asking
questions, in particular why all the F5 traffic is coming over just one IP
address in the pool.
In case folks wonder what the IP's are in the file descriptor output....
1310 Socket 898 6044* 2806 40.71.12.224:443
593a6510-ebfc-4d6b-a8f0-a0411dfee098.ods.opinsights.azure.com:443 (this is
Squid forwarding Windows event/perf data from an agent to Azure Log
Analytics)
1311 Socket 899 3015* 9208 10.26.25.220:61088 Reading next
request (10.26.25.220) is the pool IP address of the F5 in use)
1312 Socket 899 2690* 8826 10.26.25.220:61436 Reading next
request
1313 Socket 899 9169* 2884 104.208.163.218:443
eus2-jobruntimedata-prod-su1.azure-automation.net:443 (Squid to Azure)
1314 Socket 899 8787* 2508 104.208.163.218:443
eus2-jobruntimedata-prod-su1.azure-automation.net:443
1315 Socket 118 119* 3924 10.26.25.220:52153 Idle client:
Waiting for next request
1316 Socket 900 1382* 8697 10.26.25.220:54786 Reading next
request
This is from a box that restarts squid every few minutes. Typical cache.log
snippet
2019/02/26 21:24:22 kid1| storeDirWriteCleanLogs: Starting...
2019/02/26 21:24:22 kid1| Finished. Wrote 0 entries.
2019/02/26 21:24:22 kid1| Took 0.00 seconds ( 0.00 entries/sec).
2019/02/26 21:24:26 kid1| Set Current Directory to /var/cache/squid
2019/02/26 21:24:26 kid1| Starting Squid Cache version 3.5.28 for
x86_64-unknown-cygwin...
2019/02/26 21:24:26 kid1| Service Name: squid
2019/02/26 21:24:26 kid1| Process ID 1796
2019/02/26 21:24:26 kid1| Process Roles: worker
2019/02/26 21:24:26 kid1| With 3200 file descriptors available
2019/02/26 21:24:26 kid1| Initializing IP Cache...
2019/02/26 21:24:26 kid1| parseEtcHosts: /etc/hosts: (2) No such file or
directory
2019/02/26 21:24:26 kid1| DNS Socket created at [::], FD 5
2019/02/26 21:24:26 kid1| DNS Socket created at 0.0.0.0, FD 6
2019/02/26 21:24:26 kid1| Adding nameserver 208.67.220.220 from squid.conf
2019/02/26 21:24:26 kid1| Adding nameserver 208.67.222.222 from squid.conf
2019/02/26 21:24:26 kid1| Logfile: opening log
daemon:/var/log/squid/access.log
2019/02/26 21:24:26 kid1| Logfile Daemon: opening log
/var/log/squid/access.log
2019/02/26 21:24:26 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument
2019/02/26 21:24:26 kid1| Store logging disabled
2019/02/26 21:24:26 kid1| Swap maxSize 3072000 + 262144 KB, estimated 256472
objects
2019/02/26 21:24:26 kid1| Target number of buckets: 12823
2019/02/26 21:24:26 kid1| Using 16384 Store buckets
2019/02/26 21:24:26 kid1| Max Mem size: 262144 KB
2019/02/26 21:24:26 kid1| Max Swap size: 3072000 KB
2019/02/26 21:24:26 kid1| Rebuilding storage in /cygdrive/e/squid/cache
(clean log)
2019/02/26 21:24:26 kid1| Using Least Load store dir selection
2019/02/26 21:24:26 kid1| Set Current Directory to /var/cache/squid
2019/02/26 21:24:26 kid1| Finished loading MIME types and icons.
2019/02/26 21:24:26 kid1| HTCP Disabled.
2019/02/26 21:24:26 kid1| Squid plugin modules loaded: 0
2019/02/26 21:24:26 kid1| Adaptation support is off.
2019/02/26 21:24:26 kid1| Accepting HTTP Socket connections at
local=10.26.24.65:3128 remote=[::] FD 12 flags=9
2019/02/26 21:24:26 kid1| Done reading /cygdrive/e/squid/cache swaplog (0
entries)
2019/02/26 21:24:26 kid1| Store rebuilding is 0.00% complete
2019/02/26 21:24:26 kid1| Finished rebuilding storage from disk.
2019/02/26 21:24:26 kid1| 0 Entries scanned
2019/02/26 21:24:26 kid1| 0 Invalid entries.
2019/02/26 21:24:26 kid1| 0 With invalid flags.
2019/02/26 21:24:26 kid1| 0 Objects loaded.
2019/02/26 21:24:26 kid1| 0 Objects expired.
2019/02/26 21:24:26 kid1| 0 Objects cancelled.
2019/02/26 21:24:26 kid1| 0 Duplicate URLs purged.
2019/02/26 21:24:26 kid1| 0 Swapfile clashes avoided.
2019/02/26 21:24:26 kid1| Took 0.05 seconds ( 0.00 objects/sec).
2019/02/26 21:24:26 kid1| Beginning Validation Procedure
2019/02/26 21:24:27 kid1| Completed Validation Procedure
2019/02/26 21:24:27 kid1| Validated 0 Entries
2019/02/26 21:24:27 kid1| store_swap_size = 0.00 KB
2019/02/26 21:24:27 kid1| storeLateRelease: released 0 objects
2019/02/26 21:25:02 kid1| FD 12, 10.26.24.65 [ job1]: (14) Bad address
2019/02/26 21:25:02 kid1| FD 12, 10.26.24.65 [Stopped, reason:Listener
socket closed job1]: (14) Bad address
2019/02/26 21:25:02 kid1| FD 12, 10.26.24.65 [Stopped, reason:Listener
socket closed job1]: (14) Bad address
2019/02/26 21:25:02 kid1| comm_poll: poll failure: (14) Bad address
2019/02/26 21:25:02 kid1| Select loop Error. Retry 1
2019/02/26 21:25:02 kid1| comm_poll: poll failure: (14) Bad address
2019/02/26 21:25:02 kid1| Select loop Error. Retry 2
2019/02/26 21:25:02 kid1| comm_poll: poll failure: (14) Bad address
2019/02/26 21:25:02 kid1| Select loop Error. Retry 3
2019/02/26 21:25:02 kid1| comm_poll: poll failure: (14) Bad address
2019/02/26 21:25:02 kid1| Select loop Error. Retry 4
2019/02/26 21:25:02 kid1| comm_poll: poll failure: (14) Bad address
2019/02/26 21:25:02 kid1| Select loop Error. Retry 5
2019/02/26 21:25:02 kid1| comm_poll: poll failure: (14) Bad address
2019/02/26 21:25:02 kid1| Select loop Error. Retry 6
2019/02/26 21:25:02 kid1| comm_poll: poll failure: (14) Bad address
2019/02/26 21:25:02 kid1| Select loop Error. Retry 7
2019/02/26 21:25:02 kid1| comm_poll: poll failure: (14) Bad address
2019/02/26 21:25:02 kid1| Select loop Error. Retry 8
2019/02/26 21:25:02 kid1| comm_poll: poll failure: (14) Bad address
2019/02/26 21:25:02 kid1| Select loop Error. Retry 9
2019/02/26 21:25:02 kid1| comm_poll: poll failure: (14) Bad address
2019/02/26 21:25:02 kid1| Select loop Error. Retry 10
2019/02/26 21:25:02 kid1| Closing HTTP port 10.26.24.65:3128
FATAL: Event loop exited with failure.
Squid Cache (Version 3.5.28): Terminated abnormally.
CPU Usage: 13.187 seconds = 4.625 user + 8.562 sys Maximum Resident Size:
5276416 KB Page faults with physical i/o: 20822
-----Original Message-----
From: Eliezer Croitoru <eliezer at ngtech.co.il>
Sent: Tuesday, February 26, 2019 5:23 PM
To: Rafael Akchurin <rafael.akchurin at diladele.com>
Cc: Van Order, Drew (US - Hermitage) <dvanorder at deloitte.com>; 'Amos
Jeffries' <squid3 at treenet.co.nz>; squid-users at lists.squid-cache.org
Subject: [EXT] Re: [squid-users] Squid for Windows Repeatedly Crashing
It depends on the hardware in the server grade Windows.
It can take more then 3k conn's for 100%.
It's possible that squid was not designed for windows 2k16....
Eliezer
On 2019-02-24 15:47, Rafael Akchurin wrote:
> As far as I know the internal FD limit for Windows build is around 3K
> - might be being existed and thus unexpected behavior raising its ugly
> head..
>
> -----Original Message-----
> From: squid-users <squid-users-bounces at lists.squid-cache.org> On
> Behalf Of Van Order, Drew (US - Hermitage)
> Sent: Sunday, 24 February 2019 14:40
> To: eliezer at ngtech.co.il; 'Amos Jeffries' <squid3 at treenet.co.nz>;
> squid-users at lists.squid-cache.org
> Subject: Re: [squid-users] Squid for Windows Repeatedly Crashing
>
> This is helpful, and I especially appreciate the time given it is the
> weekend.
> <SNIP>
This message (including any attachments) contains confidential information
intended for a specific individual and purpose, and is protected by law. If
you are not the intended recipient, you should delete this message and any
disclosure, copying, or distribution of this message, or the taking of any
action based on it, by you is strictly prohibited.
v.E.1
_______________________________________________
squid-users mailing list
squid-users at lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users
More information about the squid-users
mailing list