[squid-users] Squid for Windows Repeatedly Crashing
Van Order, Drew (US - Hermitage)
dvanorder at deloitte.com
Thu Feb 21 05:23:25 UTC 2019
Hello folks, we're running Squid 3.5.28 on four Windows 2016 VM's, each has 4 CPU, 8 GB memory, 10 GB NIC. We implemented Squid to support forwarding Azure Log Analytics data, it's all CONNECT. The Squids are load balanced through a F5. There are less than 1,000 servers sending data to Log Analytics.
All four Squids are regularly crashing, and I don't know how to interpret the errors in cache.log. It crashes if we disable caching too.
Any insight is appreciated-I've inherited this responsibility and more a cloud engineer than a network specialist.
Thanks in advance!
Typical error sequence in cache.log
2019/02/20 09:42:32 kid1| FD 12, 10.5.11.12 [Stopped, reason:Listener socket closed job1]: (14) Bad address
2019/02/20 09:42:32 kid1| FD 12, 10.5.11.12 [Stopped, reason:Listener socket closed job1]: (14) Bad address
2019/02/20 09:42:32 kid1| FD 12, 10.5.11.12 [Stopped, reason:Listener socket closed job1]: (14) Bad address
2019/02/20 09:42:32 kid1| FD 12, 10.5.11.12 [Stopped, reason:Listener socket closed job1]: (14) Bad address
2019/02/20 09:42:32 kid1| FD 12, 10.5.11.12 [Stopped, reason:Listener socket closed job1]: (14) Bad address
2019/02/20 09:42:32 kid1| FD 12, 10.5.11.12 [Stopped, reason:Listener socket closed job1]: (14) Bad address
2019/02/20 09:42:32 kid1| FD 12, 10.5.11.12 [Stopped, reason:Listener socket closed job1]: (14) Bad address
2019/02/20 09:42:33 kid1| FD 12, 10.5.11.12 [Stopped, reason:Listener socket closed job1]: (14) Bad address
2019/02/20 09:42:33 kid1| FD 12, 10.5.11.12 [Stopped, reason:Listener socket closed job1]: (14) Bad address
2019/02/20 09:42:33 kid1| FD 12, 10.5.11.12 [Stopped, reason:Listener socket closed job1]: (14) Bad address
2019/02/20 09:42:33 kid1| FD 12, 10.5.11.12 [Stopped, reason:Listener socket closed job1]: (14) Bad address
2019/02/20 09:42:33 kid1| FD 12, 10.5.11.12 [Stopped, reason:Listener socket closed job1]: (14) Bad address
2019/02/20 09:42:33 kid1| comm_poll: poll failure: (14) Bad address
2019/02/20 09:42:33 kid1| Select loop Error. Retry 1
2019/02/20 09:42:33 kid1| comm_poll: poll failure: (14) Bad address
2019/02/20 09:42:33 kid1| Select loop Error. Retry 2
2019/02/20 09:42:33 kid1| comm_poll: poll failure: (14) Bad address
2019/02/20 09:42:33 kid1| Select loop Error. Retry 3
2019/02/20 09:42:33 kid1| comm_poll: poll failure: (14) Bad address
2019/02/20 09:42:33 kid1| Select loop Error. Retry 4
2019/02/20 09:42:33 kid1| comm_poll: poll failure: (14) Bad address
2019/02/20 09:42:33 kid1| Select loop Error. Retry 5
2019/02/20 09:42:33 kid1| comm_poll: poll failure: (14) Bad address
2019/02/20 09:42:33 kid1| Select loop Error. Retry 6
2019/02/20 09:42:33 kid1| comm_poll: poll failure: (14) Bad address
2019/02/20 09:42:33 kid1| Select loop Error. Retry 7
2019/02/20 09:42:33 kid1| comm_poll: poll failure: (14) Bad address
2019/02/20 09:42:33 kid1| Select loop Error. Retry 8
2019/02/20 09:42:33 kid1| comm_poll: poll failure: (14) Bad address
2019/02/20 09:42:33 kid1| Select loop Error. Retry 9
2019/02/20 09:42:33 kid1| comm_poll: poll failure: (14) Bad address
2019/02/20 09:42:33 kid1| Select loop Error. Retry 10
2019/02/20 09:42:33 kid1| Closing HTTP port 10.5.11.12:3128
FATAL: Event loop exited with failure.
Squid Cache (Version 3.5.28): Terminated abnormally.
CPU Usage: 12.640 seconds = 4.234 user + 8.406 sys
Maximum Resident Size: 5159680 KB
Page faults with physical i/o: 20341
Squid restarts, and will often start erroring right away:
2019/02/20 09:42:33 kid1| storeDirWriteCleanLogs: Starting...
2019/02/20 09:42:33 kid1| Finished. Wrote 0 entries.
2019/02/20 09:42:33 kid1| Took 0.00 seconds ( 0.00 entries/sec).
2019/02/20 09:42:36 kid1| Set Current Directory to /cygdrive/e/squid/var/coredump
2019/02/20 09:42:36 kid1| Starting Squid Cache version 3.5.28 for x86_64-unknown-cygwin...
2019/02/20 09:42:36 kid1| Service Name: squid
2019/02/20 09:42:36 kid1| Process ID 2292
2019/02/20 09:42:36 kid1| Process Roles: worker
2019/02/20 09:42:36 kid1| With 3200 file descriptors available
2019/02/20 09:42:36 kid1| Initializing IP Cache...
2019/02/20 09:42:36 kid1| parseEtcHosts: /etc/hosts: (2) No such file or directory
2019/02/20 09:42:36 kid1| DNS Socket created at [::], FD 5
2019/02/20 09:42:36 kid1| DNS Socket created at 0.0.0.0, FD 6
2019/02/20 09:42:36 kid1| Adding nameserver 208.67.220.220 from squid.conf
2019/02/20 09:42:36 kid1| Adding nameserver 208.67.222.222 from squid.conf
2019/02/20 09:42:36 kid1| Logfile: opening log daemon:/var/log/squid/access.log
2019/02/20 09:42:36 kid1| Logfile Daemon: opening log /var/log/squid/access.log
2019/02/20 09:42:36 kid1| WARNING: no_suid: setuid(0): (22) Invalid argument
2019/02/20 09:42:36 kid1| Store logging disabled
2019/02/20 09:42:36 kid1| Swap maxSize 262144 + 262144 KB, estimated 40329 objects
2019/02/20 09:42:36 kid1| Target number of buckets: 2016
2019/02/20 09:42:36 kid1| Using 8192 Store buckets
2019/02/20 09:42:36 kid1| Max Mem size: 262144 KB
2019/02/20 09:42:36 kid1| Max Swap size: 262144 KB
2019/02/20 09:42:36 kid1| Rebuilding storage in /cygdrive/e/squid/cache (clean log)
2019/02/20 09:42:36 kid1| Using Least Load store dir selection
2019/02/20 09:42:36 kid1| Set Current Directory to /cygdrive/e/squid/var/coredump
2019/02/20 09:42:36 kid1| Finished loading MIME types and icons.
2019/02/20 09:42:36 kid1| HTCP Disabled.
2019/02/20 09:42:36 kid1| Squid plugin modules loaded: 0
2019/02/20 09:42:36 kid1| Adaptation support is off.
2019/02/20 09:42:36 kid1| Accepting HTTP Socket connections at local=10.5.11.12:3128 remote=[::] FD 12 flags=9
2019/02/20 09:42:36 kid1| Done reading /cygdrive/e/squid/cache swaplog (0 entries)
2019/02/20 09:42:36 kid1| Store rebuilding is 0.00% complete
2019/02/20 09:42:36 kid1| Finished rebuilding storage from disk.
2019/02/20 09:42:36 kid1| 0 Entries scanned
2019/02/20 09:42:36 kid1| 0 Invalid entries.
2019/02/20 09:42:36 kid1| 0 With invalid flags.
2019/02/20 09:42:36 kid1| 0 Objects loaded.
2019/02/20 09:42:36 kid1| 0 Objects expired.
2019/02/20 09:42:36 kid1| 0 Objects cancelled.
2019/02/20 09:42:36 kid1| 0 Duplicate URLs purged.
2019/02/20 09:42:36 kid1| 0 Swapfile clashes avoided.
2019/02/20 09:42:36 kid1| Took 0.04 seconds ( 0.00 objects/sec).
2019/02/20 09:42:36 kid1| Beginning Validation Procedure
2019/02/20 09:42:36 kid1| Completed Validation Procedure
2019/02/20 09:42:36 kid1| Validated 0 Entries
2019/02/20 09:42:36 kid1| store_swap_size = 0.00 KB
2019/02/20 09:42:37 kid1| storeLateRelease: released 0 objects
2019/02/20 09:42:45 kid1| FD 12, 10.5.11.12 [ job1]: (14) Bad address
Squid.conf
# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
#acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
#acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC 4193 local private network range
acl localnet src fe80::/10 # RFC 4291 link-local (directly plugged) machines
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager
# Deny requests to certain unsafe ports
http_access deny !Safe_ports
# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports
# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
http_access deny to_localhost
# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost
# And finally deny all other access to this proxy
http_access deny all
# Squid normally listens to port 3128
http_port 10.5.11.12:3128
# Uncomment the line below to enable disk caching - path format is /cygdrive/<full path to cache folder>, i.e.
cache_dir aufs /cygdrive/e/squid/cache 256 8 64
#cache deny all
# Limit number of days to keep logs
logfile_rotate 2
# Coredump directory
coredump_dir /cygdrive/e/squid/var/coredump
# Add any of your own refresh_pattern entries above these.
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
dns_nameservers 208.67.220.220 208.67.222.222
max_filedescriptors 3200
Typical access.log
1550724138.034 213 10.27.18.220 TCP_TUNNEL/200 4301 CONNECT ac3d8ead-8d6e-423e-8f51-1beadafe281a.ods.opinsights.azure.com:443 - HIER_DIRECT/40.71.12.224 -
1550724153.063 122686 10.27.18.220 TCP_TUNNEL/200 8820 CONNECT eus2-jobruntimedata-prod-su1.azure-automation.net:443 - HIER_DIRECT/104.208.163.218 -
1550724155.287 635036 10.27.18.220 TCP_TUNNEL/200 11107 CONNECT 593a6510-ebfc-4d6b-a8f0-a0411dfee098.ods.opinsights.azure.com:443 - HIER_DIRECT/40.71.12.224 -
1550724155.299 455045 10.27.18.220 TCP_TUNNEL/200 9280 CONNECT 593a6510-ebfc-4d6b-a8f0-a0411dfee098.ods.opinsights.azure.com:443 - HIER_DIRECT/40.71.12.224 -
1550724158.005 187 10.27.18.220 TCP_TUNNEL/200 4301 CONNECT ac3d8ead-8d6e-423e-8f51-1beadafe281a.ods.opinsights.azure.com:443 - HIER_DIRECT/40.71.12.224 -
1550724178.345 505 10.27.18.220 TCP_TUNNEL/200 4301 CONNECT ac3d8ead-8d6e-423e-8f51-1beadafe281a.ods.opinsights.azure.com:443 - HIER_DIRECT/40.71.12.224 -
Andrew Van Order
CTO | Application Delivery Services | Hosting Services - Monitoring and Configuration Services
Deloitte Services LP
Tel/Direct: +1 615 882 7836 | Fax: +1 615 750 7836
dvanorder at deloitte.com | www.deloitte.com
This message (including any attachments) contains confidential information intended for a specific individual and purpose, and is protected by law. If you are not the intended recipient, you should delete this message and any disclosure, copying, or distribution of this message, or the taking of any action based on it, by you is strictly prohibited.
v.E.1
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-users/attachments/20190221/1401a615/attachment-0001.html>
More information about the squid-users
mailing list