[squid-users] SSL-Bump: NAT/TPROXY lookup failed to locate original IPs
Eliezer Croitoru
eliezer at ngtech.co.il
Sun Feb 26 18:51:01 UTC 2017
Hey Michael,
The details you attached explained pretty well the cause for the issues you have described.
What you will need to do in order to make this setup to work can be done in more then one way.
For a sysadmin the simplest way is to create a VPN or some kind of a tunnel between the AWS instance to the local router.
I am almost sure that you can use haproxy to do a local tproxy or interception that will forward the traffic to the remote squid with the PROXY protocol keeping original source and original destination visible to the remote squid.
The choice will depend on both:
- your skills and will to dig some time about couple subjects
- The availability of static IP addresses(both local and AWS).
- The OS on both sides
I believe that the next haproxy settings can be used as a compromise to a tunnel:
http://ngtech.co.il/paste/1605/
And some tproxy route and iptables rules ..
With a squid.conf which will be similar to:
acl frontend src 100.0.0.1
proxy_protocol_access allow frontend
http_port 3127
http_port 3128 require-proxy-header ... ssl-bump settings
##END of example
However I do still believe that the more secure way would be to use some kind of vpn tunnel like OpenVPN between the local router to the remote AWS instance.
All The Bests,
Eliezer
----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer at ngtech.co.il
-----Original Message-----
From: Test User [mailto:tuser6485 at gmail.com]
Sent: Sunday, February 26, 2017 8:38 AM
To: Eliezer Croitoru <eliezer at ngtech.co.il>
Cc: squid-users at lists.squid-cache.org
Subject: Re: [squid-users] SSL-Bump: NAT/TPROXY lookup failed to locate original IPs
On Sun, Feb 26, 2017 at 10:40 AM, Eliezer Croitoru <eliezer at ngtech.co.il> wrote:
> Hey Michael,
>
> You will need to clear out couple things for us.
> First we will need one of the next ouputs or both:
> iptables-save
> iptables -L -nv
>
> And then clear out where is this proxy sittings and the network structure.
> It's not clear if the squid box is the router or a machine somewhere on AWS.
> If you wish to pass traffic from a local router to a one on AWS you will need to create a tunnel like using OpenVPN or a similar solution and to use some routing rules to pass the traffic from the local LAN to AWS without removing the original destination address.
>
> When more details on the setup will be available it will be much simpler to understand what is the root for some of the issues you are having.
>
> All The Bests,
> Eliezer
>
> ----
> Eliezer Croitoru
> Linux System Administrator
> Mobile: +972-5-28704261
> Email: eliezer at ngtech.co.il
>
>
> -----Original Message-----
> From: squid-users [mailto:squid-users-bounces at lists.squid-cache.org] On Behalf Of Test User
> Sent: Friday, February 24, 2017 8:52 AM
> To: squid-users at lists.squid-cache.org
> Subject: [squid-users] SSL-Bump: NAT/TPROXY lookup failed to locate original IPs
>
> Hi,
> Sorry I am asking this question again. I am trying to setup HTTPS
> proxy using ssl-bump. I have followed
> steps mentioned in:
> http://wiki.squid-cache.org/ConfigExamples/Intercept/SslBumpExplicit
>
> Following are Squid setup details:
>
> Squid Cache: Version 3.5.12
> Service Name: squid
> Ubuntu linux
>
> configure options: '--build=x86_64-linux-gnu' '--prefix=/usr'
> '--includedir=${prefix}/include' '--mandir=${prefix}/share/man'
> '--infodir=${prefix}/share/info' '--sysconfdir=/etc'
> '--localstatedir=/var' '--libexecdir=${prefix}/lib/squid3'
> '--srcdir=.' '--disable-maintainer-mode'
> '--disable-dependency-tracking' '--disable-silent-rules'
> 'BUILDCXXFLAGS=-g -O2 -fPIE -fstack-protector-strong -Wformat
> -Werror=format-security -Wl,-Bsymbolic-functions -fPIE -pie
> -Wl,-z,relro -Wl,-z,now' '--datadir=/usr/share/squid'
> '--sysconfdir=/etc/squid' '--libexecdir=/usr/lib/squid'
> '--mandir=/usr/share/man' '--enable-inline' '--disable-arch-native'
> '--enable-async-io=8' '--enable-storeio=ufs,aufs,diskd,rock'
> '--enable-removal-policies=lru,heap' '--enable-delay-pools'
> '--enable-cache-digests' '--enable-icap-client'
> '--enable-follow-x-forwarded-for'
> '--enable-auth-basic=DB,fake,getpwnam,LDAP,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB'
> '--enable-auth-digest=file,LDAP'
> '--enable-auth-negotiate=kerberos,wrapper'
> '--enable-auth-ntlm=fake,smb_lm'
> '--enable-external-acl-helpers=file_userip,kerberos_ldap_group,LDAP_group,session,SQL_session,unix_group,wbinfo_group'
> '--enable-url-rewrite-helpers=fake' '--enable-eui' '--enable-esi'
> '--enable-icmp' '--enable-zph-qos' '--enable-ecap' '--with-openssl'
> '--enable-ssl-crtd' '--disable-translation'
> '--with-swapdir=/var/spool/squid' '--with-logdir=/var/log/squid'
> '--with-pidfile=/var/run/squid.pid' '--with-filedescriptors=65536'
> '--with-large-files' '--with-default-user=proxy'
> '--enable-build-info=Ubuntu linux' '--enable-linux-netfilter'
> 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fPIE
> -fstack-protector-strong -Wformat -Werror=format-security -Wall'
> 'LDFLAGS=-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now'
> 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 -fPIE
> -fstack-protector-strong -Wformat -Werror=format-security'
>
>
> Following is my squid.conf file:
>
> acl SSL_ports port 443
> acl Safe_ports port 80 # http
> acl Safe_ports port 21 # ftp
> acl Safe_ports port 443 # https
> acl Safe_ports port 70 # gopher
> acl Safe_ports port 210 # wais
> acl Safe_ports port 1025-65535 # unregistered ports
> acl Safe_ports port 280 # http-mgmt
> acl Safe_ports port 488 # gss-http
> acl Safe_ports port 591 # filemaker
> acl Safe_ports port 777 # multiling http
> acl CONNECT method CONNECT
> acl step1 at_step SslBump1
> http_access deny !Safe_ports
> http_access deny CONNECT !SSL_ports
> http_access allow localhost manager
> http_access deny manager
> http_access allow localhost
> http_access allow all
> http_port 3128 ssl-bump \
> cert=/etc/squid/ssl_cert/squidCA.pem \
> generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
> https_port 3129 intercept ssl-bump generate-host-certificates=on \
> dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_cert/squidCA.pem \
> dhparams=/etc/squid/ssl_cert/dhparam.pem
> sslproxy_options NO_SSLv2,NO_SSLv3,SINGLE_DH_USE
> sslproxy_cipher
> EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS
> sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/spool/squid3_ssldb -M 4MB
> debug_options ALL,1 3,5 4,5 11,5 17,5 23,5 46,5 78,5 rotate=1
> coredump_dir /var/spool/squid
> refresh_pattern ^ftp: 1440 20% 10080
> refresh_pattern ^gopher: 1440 0% 1440
> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
> refresh_pattern (Release|Packages(.gz)*)$ 0 20% 2880
> refresh_pattern . 0 20% 4320
>
>
> I get no errors while starting Squid. Following are the logs when Squid starts:
>
> 2017/02/23 09:59:53 kid1| Set Current Directory to /var/spool/squid
> 2017/02/23 09:59:53 kid1| Starting Squid Cache version 3.5.12 for
> x86_64-pc-linux-gnu...
> 2017/02/23 09:59:53 kid1| Service Name: squid
> 2017/02/23 09:59:53 kid1| Process ID 26236
> 2017/02/23 09:59:53 kid1| Process Roles: worker
> 2017/02/23 09:59:53 kid1| With 65535 file descriptors available
> 2017/02/23 09:59:53 kid1| Initializing IP Cache...
> 2017/02/23 09:59:53.756 kid1| 78,2| dns_internal.cc(1525) dnsInit:
> idnsInit: attempt open DNS socket to: [::]
> 2017/02/23 09:59:53.756 kid1| 78,2| dns_internal.cc(1534) dnsInit:
> idnsInit: attempt open DNS socket to: 0.0.0.0
> 2017/02/23 09:59:53.756 kid1| DNS Socket created at [::], FD 6
> 2017/02/23 09:59:53.756 kid1| DNS Socket created at 0.0.0.0, FD 7
> 2017/02/23 09:59:53.756 kid1| Adding nameserver 172.31.0.2 from /etc/resolv.conf
> 2017/02/23 09:59:53.756 kid1| 78,3| dns_internal.cc(321)
> idnsAddNameserver: idnsAddNameserver: Added nameserver #0
> (172.31.0.2:53)
> 2017/02/23 09:59:53.756 kid1| Adding domain
> ap-south-1.compute.internal from /etc/resolv.conf
> 2017/02/23 09:59:53.756 kid1| 78,3| dns_internal.cc(350)
> idnsAddPathComponent: idnsAddPathComponent: Added domain #0:
> ap-south-1.compute.internal
> 2017/02/23 09:59:53.756 kid1| helperOpenServers: Starting 5/32
> 'ssl_crtd' processes
> 2017/02/23 09:59:53.775 kid1| 46,2| Format.cc(64) parse: got
> definition '%>a/%>A %un %>rm myip=%la myport=%lp'
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for
> possible Misc token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for
> possible 2C token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(389) parse: scan for
> possible 1C token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for
> possible Misc token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for
> possible 2C token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(389) parse: scan for
> possible 1C token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for
> possible Misc token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for
> possible 2C token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for
> possible Misc token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for
> possible 2C token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for
> possible Misc token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for
> possible 2C token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for
> possible Misc token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for
> possible 2C token
> 2017/02/23 09:59:53.775 kid1| 46,2| Format.cc(64) parse: got
> definition '%>a/%>A %un %>rm myip=%la myport=%lp'
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for
> possible Misc token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for
> possible 2C token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(389) parse: scan for
> possible 1C token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for
> possible Misc token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for
> possible 2C token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(389) parse: scan for
> possible 1C token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for
> possible Misc token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for
> possible 2C token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for
> possible Misc token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for
> possible 2C token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for
> possible Misc token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for
> possible 2C token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(380) parse: scan for
> possible Misc token
> 2017/02/23 09:59:53.775 kid1| 46,5| Token.cc(384) parse: scan for
> possible 2C token
> 2017/02/23 09:59:53.775 kid1| Logfile: opening log
> daemon:/var/log/squid/access.log
> 2017/02/23 09:59:53.775 kid1| Logfile Daemon: opening log
> /var/log/squid/access.log
> 2017/02/23 09:59:53.779 kid1| 23,5| url.cc(43) urlInitialize:
> urlInitialize: Initializing...
> 2017/02/23 09:59:53.779 kid1| Local cache digest enabled;
> rebuild/rewrite every 3600/3600 sec
> 2017/02/23 09:59:53.779 kid1| Store logging disabled
> 2017/02/23 09:59:53.779 kid1| Swap maxSize 0 + 262144 KB, estimated
> 20164 objects
> 2017/02/23 09:59:53.779 kid1| Target number of buckets: 1008
> 2017/02/23 09:59:53.779 kid1| Using 8192 Store buckets
> 2017/02/23 09:59:53.779 kid1| Max Mem size: 262144 KB
> 2017/02/23 09:59:53.779 kid1| Max Swap size: 0 KB
> 2017/02/23 09:59:53.779 kid1| Using Least Load store dir selection
> 2017/02/23 09:59:53.779 kid1| Set Current Directory to /var/spool/squid
> 2017/02/23 09:59:53.785 kid1| 23,3| url.cc(357) urlParse: urlParse:
> Split URL 'http://ip-172-31-25-235:3128/squid-internal-static/icons/silk/image.png'
> into proto='http', host='ip-172-31-25-235', port='3128',
> path='/squid-internal-static/icons/silk/image.png'
> 2017/02/23 09:59:53.785 kid1| 23,3| url.cc(357) urlParse: urlParse:
> Split URL 'http://ip-172-31-25-235:3128/squid-internal-static/icons/silk/page_white_text.png'
> into proto='http', host='ip-172-31-25-235', port='3128',
> path='/squid-internal-static/icons/silk/page_white_text.png'
>
> ****several urlParse logs like above. Removing them to shorten the
> email. Further logs below...****
>
> 2017/02/23 09:59:53.815 kid1| Finished loading MIME types and icons.
> 2017/02/23 09:59:53.815 kid1| HTCP Disabled.
> 2017/02/23 09:59:53.815 kid1| Pinger socket opened on FD 25
> 2017/02/23 09:59:53.815 kid1| Squid plugin modules loaded: 0
> 2017/02/23 09:59:53.815 kid1| Adaptation support is off.
> 2017/02/23 09:59:53.815 kid1| Accepting SSL bumped HTTP Socket
> connections at local=[::]:3128 remote=[::] FD 22 flags=9
> 2017/02/23 09:59:53.815 kid1| Accepting NAT intercepted SSL bumped
> HTTPS Socket connections at local=[::]:3129 remote=[::] FD 23 flags=41
> 2017/02/23 09:59:53| pinger: Initialising ICMP pinger ...
> 2017/02/23 09:59:53| pinger: ICMP socket opened.
> 2017/02/23 09:59:53| pinger: ICMPv6 socket opened
> 2017/02/23 09:59:54 kid1| storeLateRelease: released 0 objects
>
>
>
> I tested this setup by providing proxy details to Firefox. Firefox was
> able to show HTTP websites but when I tried to open an HTTPS website I
> got following error:
>
> 2017/02/23 11:00:50 kid1| ERROR: NF getsockopt(ORIGINAL_DST) failed on
> local=172.31.25.235:3129 remote=182.72.78.122:50655 FD 7 flags=33:
> (92) Protocol not available
> 2017/02/23 11:00:50 kid1| ERROR: NAT/TPROXY lookup failed to locate
> original IPs on local=172.31.25.235:3129 remote=182.72.78.122:50655 FD
> 7 flags=33
> 2017/02/23 11:00:50 kid1| ERROR: NF getsockopt(ORIGINAL_DST) failed on
> local=172.31.25.235:3129 remote=182.72.78.122:50656 FD 7 flags=33:
> (92) Protocol not available
> 2017/02/23 11:00:50 kid1| ERROR: NAT/TPROXY lookup failed to locate
> original IPs on local=172.31.25.235:3129 remote=182.72.78.122:50656 FD
> 7 flags=33
> 2017/02/23 11:00:50 kid1| ERROR: NF getsockopt(ORIGINAL_DST) failed on
> local=172.31.25.235:3129 remote=182.72.78.122:50657 FD 7 flags=33:
> (92) Protocol not available
> 2017/02/23 11:00:50 kid1| ERROR: NAT/TPROXY lookup failed to locate
> original IPs on local=172.31.25.235:3129 remote=182.72.78.122:50657 FD
> 7 flags=33
>
> I googled this error and found this mail thread which had similar problems:
> http://squid-web-proxy-cache.1019090.n4.nabble.com/NAT-TPROXY-lookup-failed-to-locate-original-IPs-td4675464.html
>
> I found this link from the above thread. I modified the steps for
> HTTPS from the below link:
> http://wiki.squid-cache.org/ConfigExamples/Intercept/LinuxDnat
>
> Now my sysctl.conf is:
>
> net.ipv4.conf.all.rp_filter=0
> net.ipv4.ip_forward = 1
> net.ipv4.conf.default.rp_filter = 0
> net.ipv4.conf.default.accept_source_route = 0
>
> My iptables -t nat -L result:
>
> Chain PREROUTING (policy ACCEPT)
> target prot opt source destination
> ACCEPT tcp -- ec2-35-154-101-8.ap-south-1.compute.amazonaws.com
> anywhere tcp dpt:https
> DNAT tcp -- anywhere anywhere tcp
> dpt:https to:35.154.101.8:3129
>
> Chain INPUT (policy ACCEPT)
> target prot opt source destination
>
> Chain OUTPUT (policy ACCEPT)
> target prot opt source destination
>
> Chain POSTROUTING (policy ACCEPT)
> target prot opt source destination
> MASQUERADE all -- anywhere anywhere
>
>
> Once this was done, I tried to hit HTTPS website from Firefox and now
> I get connection timeout error. Nothing shows in syslog, access.log or
> cache.log. Could you please help me resolve this.
>
> Thanks,
> Michael
> _______________________________________________
> squid-users mailing list
> squid-users at lists.squid-cache.org
> http://lists.squid-cache.org/listinfo/squid-users
>
Thanks for replying Eliezer. Following are the outputs you asked:
1. iptables-save:
# Generated by iptables-save v1.6.0 on Sun Feb 26 06:28:46 2017
*filter
:INPUT ACCEPT [171:12090]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [106:15187]
COMMIT
# Completed on Sun Feb 26 06:28:46 2017
# Generated by iptables-save v1.6.0 on Sun Feb 26 06:28:46 2017
*mangle
:PREROUTING ACCEPT [89003:74850371]
:INPUT ACCEPT [88973:74849159]
:FORWARD ACCEPT [30:1212]
:OUTPUT ACCEPT [76710:51478183]
:POSTROUTING ACCEPT [76740:51479395]
-A PREROUTING -p tcp -m tcp --dport 3129 -j DROP
COMMIT
# Completed on Sun Feb 26 06:28:46 2017
# Generated by iptables-save v1.6.0 on Sun Feb 26 06:28:46 2017
*nat
:PREROUTING ACCEPT [7766:436942]
:INPUT ACCEPT [7766:436942]
:OUTPUT ACCEPT [952:102330]
:POSTROUTING ACCEPT [0:0]
-A PREROUTING -s 35.154.101.8/32 -p tcp -m tcp --dport 443 -j ACCEPT
-A PREROUTING -p tcp -m tcp --dport 443 -j DNAT --to-destination
35.154.101.8:3129
-A POSTROUTING -j MASQUERADE
COMMIT
# Completed on Sun Feb 26 06:28:46 2017
2. Also pasting sudo iptables -L -nv:
Chain INPUT (policy ACCEPT 216 packets, 16058 bytes)
pkts bytes target prot opt in out source
destination
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source
destination
Chain OUTPUT (policy ACCEPT 161 packets, 24629 bytes)
pkts bytes target prot opt in out source destination
> And then clear out where is this proxy sittings and the network structure.
> It's not clear if the squid box is the router or a machine somewhere on AWS.
[Michael] This proxy is installed on an AWS instance.
> If you wish to pass traffic from a local router to a one on AWS you will need to create a tunnel like using OpenVPN or a similar solution and to use some routing rules to pass the traffic from the local LAN to AWS without removing the original destination address.
>
[Michael] Does this mean, to make ssl-bump work, I will have to setup
a VPN server and configure the VPN clients to use this proxy via VPN
server?
Thanks,
Michael.
More information about the squid-users
mailing list