[squid-users] ssl_bump problems with pypi servers
hanxie
txie145 at gmail.com
Tue May 26 23:43:53 UTC 2020
Hi all,
I am experiencing somewhat of a strange error with squid using ssl-bump. I
think I am running a somewhat typical set up in which we run a squid proxy
server fleet that is used by our other servers and we use "ssl_bump" to
man-in-the-middle all our traffic.
The problem is that occasionally requests to "https://pypi.org" will time
out. Even weirder, we are running in Google Compute Platform and the same
squid container will error out depending on different regions. This bug has
been confounding us for months now and any help would be greatly
appreciated.
We have tried turning on verbose debugging and I think I have found the logs
in which squid encounters an error with the request:
2020/05/21 19:27:28.567| 83,5| PeerConnector.cc(88) prepareSocket:
local=172.17.0.2:57012 remote=151.101.0.223:443 FD 14 flags=1,
this=0x1d3b788
2020/05/21 19:27:28.567| 83,5| PeerConnector.cc(94) prepareSocket:
local=172.17.0.2:57012 remote=151.101.0.223:443 FD 14 flags=1
2020/05/21 19:27:28.567| 9,5| AsyncCall.cc(26) AsyncCall: The AsyncCall
Security::PeerConnector::commCloseHandler constructed, this=0x1d2c990
[call2471]
2020/05/21 19:27:28.567| 5,5| comm.cc(985) comm_add_close_handler:
comm_add_close_handler: FD 14, AsyncCall=0x1d2c990*1
2020/05/21 19:27:28.567| 83,5| PeerConnector.cc(107) initialize:
local=172.17.0.2:57012 remote=151.101.0.223:443 FD 14 flags=1, ctx=0x176ea70
2020/05/21 19:27:28.567| 83,5| Session.cc(103) NewSessionObject: SSL_new
session=0x1d37d50
2020/05/21 19:27:28.567| 83,5| bio.cc(612) squid_bio_ctrl: 0x1d1a380
104(6001, 0x7ffe0c004ee4)
2020/05/21 19:27:28.567| 83,5| Session.cc(161) CreateSession: link FD 14 to
TLS session=0x1d37d50
2020/05/21 19:27:28.567| 83,5| PeerConnector.cc(123) initialize:
local=172.17.0.2:57012 remote=151.101.0.223:443 FD 14 flags=1,
session=0x1d37d50
2020/05/21 19:27:28.567| 14,3| Address.cc(382) lookupHostIP: Given Non-IP
'pypi.org': Name or service not known
2020/05/21 19:27:28.567| 83,5| BlindPeerConnector.cc(60) initialize: success
2020/05/21 19:27:28.567| 83,5| PeerConnector.cc(188) negotiate: SSL_connect
session=0x1d37d50
2020/05/21 19:27:28.567| 83,5| bio.cc(612) squid_bio_ctrl: 0x1d1a380 6(0,
0x1d16bf0)
2020/05/21 19:27:28.568| 83,5| bio.cc(113) write: FD 14 wrote 314 <= 314
2020/05/21 19:27:28.568| 83,5| bio.cc(612) squid_bio_ctrl: 0x1d1a380 11(0,
0)
2020/05/21 19:27:28.568| 83,5| bio.cc(136) read: FD 14 read -1 <= 65535
2020/05/21 19:27:28.568| 83,5| bio.cc(141) read: error: 11 ignored: 1
2020/05/21 19:27:28.568| 83,5| PeerConnector.cc(462) noteWantRead:
local=172.17.0.2:57012 remote=151.101.0.223:443 FD 14 flags=1
2020/05/21 19:27:28.568| 5,3| comm.cc(559) commSetConnTimeout:
local=172.17.0.2:57012 remote=151.101.0.223:443 FD 14 flags=1 timeout 60
2020/05/21 19:27:28.568| 5,5| ModEpoll.cc(117) SetSelect: FD 14, type=1,
handler=1, client_data=0x1d2ca60, timeout=0
2020/05/21 19:27:28.568| 93,5| AsyncJob.cc(154) callEnd:
Security::BlindPeerConnector status out: [ FD 14 job74]
2020/05/21 19:27:28.568| 93,5| AsyncCallQueue.cc(57) fireNext: leaving
AsyncJob::start()
2020/05/21 19:27:28.580| 83,5| PeerConnector.cc(188) negotiate: SSL_connect
session=0x1d37d50
2020/05/21 19:27:28.580| 83,5| bio.cc(136) read: FD 14 read 4143 <= 65535
2020/05/21 19:27:28.580| 83,5| Handshake.cc(405) parseExtensions: first
unsupported extension: 43
2020/05/21 19:27:28.580| 24,5| BinaryTokenizer.cc(47) want: 1 more bytes for
TLSPlaintext.type occupying 1 bytes @4143 in 0x1b94840;
2020/05/21 19:27:28.580| 83,5| Handshake.cc(533) parseHello: need more data
2020/05/21 19:27:28.580| 83,5| PeerConnector.cc(462) noteWantRead:
local=172.17.0.2:57012 remote=151.101.0.223:443 FD 14 flags=1
2020/05/21 19:27:28.580| 5,3| comm.cc(559) commSetConnTimeout:
local=172.17.0.2:57012 remote=151.101.0.223:443 FD 14 flags=1 timeout 60
2020/05/21 19:27:28.580| 5,5| ModEpoll.cc(117) SetSelect: FD 14, type=1,
handler=1, client_data=0x1d18730, timeout=0
2020/05/21 19:27:29.129| 41,5| AsyncCall.cc(26) AsyncCall: The AsyncCall
logfileFlush constructed, this=0x1d018e0 [call2473]
2020/05/21 19:27:29.129| 41,5| AsyncCall.cc(93) ScheduleCall: event.cc(241)
will call logfileFlush(0x1895498*?) [call2473]
2020/05/21 19:27:29.129| 41,5| AsyncCallQueue.cc(55) fireNext: entering
logfileFlush(0x1895498*?)
2020/05/21 19:27:29.129| 41,5| AsyncCall.cc(38) make: make call logfileFlush
[call2473]
2020/05/21 19:27:29.129| 41,5| AsyncCallQueue.cc(57) fireNext: leaving
logfileFlush(0x1895498*?)
2020/05/21 19:27:29.129| 41,5| AsyncCall.cc(26) AsyncCall: The AsyncCall
MaintainSwapSpace constructed, this=0x1d018e0 [call2474]
2020/05/21 19:27:29.129| 41,5| AsyncCall.cc(93) ScheduleCall: event.cc(241)
will call MaintainSwapSpace() [call2474]
2020/05/21 19:27:29.129| 41,5| AsyncCallQueue.cc(55) fireNext: entering
MaintainSwapSpace()
2020/05/21 19:27:29.129| 41,5| AsyncCall.cc(38) make: make call
MaintainSwapSpace [call2474]
2020/05/21 19:27:29.129| 41,5| AsyncCallQueue.cc(57) fireNext: leaving
MaintainSwapSpace()
2020/05/21 19:27:29.651| 41,5| AsyncCall.cc(26) AsyncCall: The AsyncCall
memPoolCleanIdlePools constructed, this=0x1d018e0 [call2475]
We are running squid 4.9 with the following configs:
# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 0.0.0.1-0.255.255.255 # RFC 1122 "this" network (LAN)
acl localnet src 10.0.0.0/8 # RFC 1918 local private network
(LAN)
acl localnet src 100.64.0.0/10 # RFC 6598 shared address space
(CGN)
acl localnet src 169.254.0.0/16 # RFC 3927 link-local (directly
plugged) machines
acl localnet src 172.16.0.0/12 # RFC 1918 local private network
(LAN)
acl localnet src 192.168.0.0/16 # RFC 1918 local private network
(LAN)
acl localnet src fc00::/7 # RFC 4193 local private network
range
acl localnet src fe80::/10 # RFC 4291 link-local (directly
plugged) machines
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
acl google_bq_storage ssl::server_name bigquerystorage.googleapis.com
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
### Custom rules follows: ###
# Logging rules
cache_log /var/log/squid/cache.log
access_log daemon:/var/log/squid/access.log squid
# Intercept all egress and man-the-middle with our own certificate
ssl_bump splice google_bq_storage
ssl_bump bump all
http_port 3128 ssl-bump tls-cert=/etc/squid/ssl_cert/our_cert.pem
tls-key=/etc/squid/ssl_cert/our_cert.pem
options=SINGLE_DH_USE,SINGLE_ECDH_USE
sslcrtd_program
/nix/store/3pwrqm8fjf6bljp3rk50bk5n3ziwaz0a-squid-4.9/libexec/security_file_certgen
-s
/nix/store/3pwrqm8fjf6bljp3rk50bk5n3ziwaz0a-squid-4.9/var/cache/squid/ssl_db
-M 4MB
# Additional certificate authorities to trust. Used when squid establishes
TLS with the target site.
tls_outgoing_options cafile=/etc/squid/ssl_cert/additional_root_certs.crt
cache deny all
### Custom rules end: ###
coredump_dir /squid/var/cache/squid
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
--
Sent from: http://squid-web-proxy-cache.1019090.n4.nabble.com/Squid-Users-f1019091.html
More information about the squid-users
mailing list