<div dir="ltr">Hi all,<br><br><div class="gmail-post-text">
<p>For couple of days I'm trying to figure out how to get a transparent
HTTPs proxy to work with Squid. What I'm trying to achieve is a proxy
that accepts internet traffic from ports 80 & 443, routes them
through Squid to Privoxy and finally through Tor and returns back the
data. So essentially I want to "automatically" revert some traffic
through Tor without the user needing to add a proxy to their connection.</p>
<p>I know how to setup the Privoxy and Tor part, but I'm struggling with the Squid & IP tables configuration.</p>
<h2>Here is my setup</h2>
<p>Download latest version</p>
<pre><code>curl -O <a href="http://www.squid-cache.org/Versions/v3/3.5/squid-3.5.22.tar.gz">http://www.squid-cache.org/Versions/v3/3.5/squid-3.5.22.tar.gz</a> && tar zxvf squid-3.5.22.tar.gz && cd squid-3.5.22
</code></pre>
<p>Install all needed packages</p>
<pre><code>apt install devscripts build-essential openssl libssl-dev fakeroot libcppunit-dev libsasl2-dev cdbs ccze libfile-readbackwards-perl libcap2 libcap-dev libcap2-dev libnetfilter-conntrack-dev htop ccze sysv-rc-conf -y
</code></pre>
<p>Configure the build and make and install</p>
<pre><code>./configure \
CHOST="x86_64-pc-linux-gnu" \
CFLAGS="-march=core2 -O2 -pipe" \
CXXFLAGS="${CFLAGS}" \
--build=x86_64-linux-gnu \
--prefix=/usr \
--exec-prefix=/usr \
--bindir=/usr/bin \
--sbindir=/usr/sbin \
--libdir=/usr/lib \
--sharedstatedir=/usr/com \
--includedir=/usr/include \
--localstatedir=/var \
--libexecdir=/usr/lib/squid \
--srcdir=. \
--datadir=/usr/share/squid \
--sysconfdir=/etc/squid \
--infodir=/usr/share/info \
--mandir=/usr/share/man \
--x-includes=/usr/include \
--x-libraries=/usr/lib \
--with-default-user=proxy \
--with-logdir=/var/log/squid \
--with-pidfile=/var/run/squid.pid \
--enable-err-languages=English \
--enable-default-err-language=English \
--enable-storeio=ufs,aufs,diskd \
--enable-linux-netfilter \
--enable-removal-policies=lru,heap \
--enable-gnuregex \
--enable-follow-x-forwarded-for \
--enable-x-accelerator-vary \
--enable-zph-qos \
--enable-delay-pools \
--enable-snmp \
--enable-underscores \
--with-openssl \
--enable-ssl-crtd \
--enable-http-violations \
--enable-async-io=24 \
--enable-storeid-rewrite-helpers \
--with-large-files \
--with-libcap \
--with-netfilter-conntrack \
--with-included-ltdl \
--with-maxfd=65536 \
--with-filedescriptors=65536 \
--with-pthreads \
--without-gnutls \
--without-mit-krb5 \
--without-heimdal-krb5 \
--without-gnugss \
--disable-icap-client \
--disable-wccp \
--disable-wccpv2 \
--disable-dependency-tracking \
--disable-auth --disable-epoll \
--disable-ident-lookups \
--disable-icmp
</code></pre>
<p>Allow ip4 forwarding</p>
<pre><code>echo -e "net.ipv4.ip_forward = 1\nnet.ipv4.conf.default.rp_filter = 0\nnet.ipv4.conf.all.rp_filter = 0\nnet.ipv4.conf.eth0.rp_filter = 0\n" >> /etc/sysctl.conf
</code></pre>
<p>Generate certificates</p>
<pre><code>mkdir /etc/squid/ssl_certs && cd /etc/squid/ssl_certs
openssl genrsa -out squid.key 2048
openssl req -new -key squid.key -out squid.csr -nodes
openssl x509 -req -days 3652 -in squid.csr -signkey squid.key -out squid.crt
cat squid.crt squid.key > squid.pem
</code></pre>
<p>Generate certificate cache</p>
<pre><code>mkdir /var/lib/squid && chown -R proxy:proxy /var/lib/squid/
/usr/lib/squid/ssl_crtd -c -s /var/lib/squid/ssl_db
</code></pre>
<p>Change ownership and rights to folders</p>
<pre><code>mkdir -p /var/spool/squid
chown -R proxy:proxy /etc/squid/squid.conf | chown -R proxy:proxy /usr/lib/squid | chown -R proxy:proxy /var/lib/squid/ssl_db/ | chown -R proxy:proxy /var/spool/squid | chown -R proxy:proxy /var/log/squid | chmod 777 /var/spool/squid | chmod 777 /var/log/squid | chmod 755 /var/lib/squid/ssl_db/certs | chown proxy:proxy /var/log/squid/
</code></pre>
<p>Change configuration (bellow) and initialize the cache</p>
<pre><code>squid -f /etc/squid/squid.conf -z
</code></pre>
<p>Redirect ports 80 and 443</p>
<pre><code>iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-ports 3128
iptables -t nat -A PREROUTING -p tcp --dport 443 -j REDIRECT --to-ports 3129
</code></pre>
<p>My actual squid configuration</p>
<pre><code>acl localnet src all
acl SSL_ports port 443
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl CONNECT method CONNECT
never_direct allow all
always_direct allow all
# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager
http_access allow localnet
http_access allow localhost
debug_options ALL,2
visible_hostname squid
# stop squid taking forever to restart.
shutdown_lifetime 3
# for clients with a configured proxy.
http_port 3127
# for clients who are sent here via iptables ... REDIRECT.
http_port 3128 tproxy
# for https clients who are sent here via iptables ... REDIRECT
https_port 3129 tproxy ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_certs/squid.pem
sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB sslcrtd_children 8 startup=1 idle=1
# acl step1 at_step SslBump1
# ssl_bump peek step1
# ssl_bump bump all
ssl_bump server-first all
sslproxy_cert_error allow all
sslproxy_flags DONT_VERIFY_PEER
via off
forwarded_for off
request_header_access From deny all
request_header_access Server deny all
request_header_access WWW-Authenticate deny all
request_header_access Link deny all
request_header_access Cache-Control deny all
request_header_access Proxy-Connection deny all
request_header_access X-Cache deny all
request_header_access X-Cache-Lookup deny all
request_header_access Via deny all
request_header_access X-Forwarded-For deny all
request_header_access Pragma deny all
request_header_access Keep-Alive deny all
cache_dir ufs /var/spool/squid 1024 16 256
coredump_dir /var/cache/squid
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern . 0 20% 4320
</code></pre>
<hr>
<p>You can notice how benevolent I'm with the settings for Squid. It's only for testing.</p>
<p>So where I got now is that nor intercept nor tproxy works. If I use
accel for the non-HTTPS traffic it works, but nothing else. If I use it
as it is, the result is that it will end up hanging for the client's
timeout period and then timeout.</p>
<p>Here is an example. I changed in <code>/etc/hosts</code> the IP for <a href="http://httpbin.org">httpbin.org</a> and redirected it through the squid box.</p>
<pre><code>❯ curl -vk <a href="https://httpbin.org/ip">https://httpbin.org/ip</a>
* Trying *******...
* Connected to <a href="http://httpbin.org">httpbin.org</a> (*******) port 443 (#0)
* TLS 1.2 connection using TLS_RSA_WITH_AES_256_GCM_SHA384
* Server certificate: ******
* Server certificate: Universe
> GET /ip HTTP/1.1
> Host: <a href="http://httpbin.org">httpbin.org</a>
> User-Agent: curl/7.49.1
> Accept: */*
>
< HTTP/1.1 503 Service Unavailable
< Server: squid/3.5.22
< Mime-Version: 1.0
< Date: Mon, 05 Dec 2016 05:43:50 GMT
< Content-Type: text/html;charset=utf-8
< Content-Length: 3498
< X-Squid-Error: ERR_CONNECT_FAIL 110
< Vary: Accept-Language
< Content-Language: en
< X-Cache: MISS from pipik
< Connection: close
</code></pre>
<p>On the squid side</p>
<pre><code>2016/12/05 05:42:50.362 kid1| 5,2| TcpAcceptor.cc(220) doAccept: New connection on FD 28
2016/12/05 05:42:50.362 kid1| 5,2| TcpAcceptor.cc(295) acceptNext: connection on local=[::]:3129 remote=[::] FD 28 flags=25
2016/12/05 05:42:50.363 kid1| 33,2| client_side.cc(3911) httpsSslBumpAccessCheckDone: sslBump needed for local=*******:3129 remote=############# FD 11 flags=17 method 3
2016/12/05 05:42:50.363 kid1| 11,2| client_side.cc(2347) parseHttpRequest: HTTP Client local=*******:3129 remote=############# FD 11 flags=17
2016/12/05 05:42:50.363 kid1| 11,2| client_side.cc(2348) parseHttpRequest: HTTP Client REQUEST:
---------
CONNECT *******:3129 HTTP/1.1
Host: *******:3129
----------
2016/12/05 05:42:50.363 kid1| 85,2| client_side_request.cc(744) clientAccessCheckDone: The request CONNECT *******:3129 is ALLOWED; last ACL checked: localnet
2016/12/05 05:42:50.363 kid1| 85,2| client_side_request.cc(720) clientAccessCheck2: No adapted_http_access configuration. default: ALLOW
2016/12/05 05:42:50.363 kid1| 85,2| client_side_request.cc(744) clientAccessCheckDone: The request CONNECT *******:3129 is ALLOWED; last ACL checked: localnet
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.378 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.379 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.379 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.379 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.379 kid1| 83,2| client_side.cc(4284) clientPeekAndSpliceSSL: SSL_accept failed.
2016/12/05 05:42:50.379 kid1| 17,2| FwdState.cc(133) FwdState: Forwarding client request local=*******:3129 remote=############# FD 11 flags=17, url=*******:3129
2016/12/05 05:42:50.379 kid1| 44,2| peer_select.cc(280) peerSelectDnsPaths: Found sources for '*******:3129'
2016/12/05 05:42:50.379 kid1| 44,2| peer_select.cc(281) peerSelectDnsPaths: always_direct = ALLOWED
2016/12/05 05:42:50.379 kid1| 44,2| peer_select.cc(282) peerSelectDnsPaths: never_direct = DUNNO
2016/12/05 05:42:50.379 kid1| 44,2| peer_select.cc(288) peerSelectDnsPaths: ORIGINAL_DST = local=############# remote=*******:3129 flags=25
2016/12/05 05:42:50.379 kid1| 44,2| peer_select.cc(295) peerSelectDnsPaths: timedout = 0
2016/12/05 05:43:50.645 kid1| 4,2| errorpage.cc(1261) BuildContent: No existing error page language negotiated for ERR_CONNECT_FAIL. Using default error file.
2016/12/05 05:43:50.645 kid1| 20,2| store.cc(980) checkCachable: StoreEntry::checkCachable: NO: not cachable
2016/12/05 05:43:50.645 kid1| 20,2| store.cc(980) checkCachable: StoreEntry::checkCachable: NO: not cachable
2016/12/05 05:43:50.845 kid1| 83,2| client_side.cc(3811) clientNegotiateSSL: clientNegotiateSSL: New session 0x29dda60 on FD 11 (#############:59117)
2016/12/05 05:43:50.943 kid1| 11,2| client_side.cc(2347) parseHttpRequest: HTTP Client local=*******:3129 remote=############# FD 11 flags=17
2016/12/05 05:43:50.944 kid1| 11,2| client_side.cc(2348) parseHttpRequest: HTTP Client REQUEST:
---------
GET /ip HTTP/1.1
Host: <a href="http://httpbin.org">httpbin.org</a>
User-Agent: curl/7.49.1
Accept: */*
----------
2016/12/05 05:43:50.944 kid1| 33,2| QosConfig.cc(145) doTosLocalMiss: QOS: Preserving TOS on miss, TOS=0
2016/12/05 05:43:50.944 kid1| 33,2| client_side_reply.cc(1534) buildReplyHeader: clientBuildReplyHeader: Connection Keep-Alive not requested by admin or client
2016/12/05 05:43:50.944 kid1| 88,2| client_side_reply.cc(2051) processReplyAccessResult: The reply for GET <a href="https://httpbin.org/ip">https://httpbin.org/ip</a> is ALLOWED, because it matched (access_log daemon:/var/log/squid/access.log line)
2016/12/05 05:43:50.944 kid1| 11,2| client_side.cc(1393) sendStartOfMessage: HTTP Client local=*******:3129 remote=############# FD 11 flags=17
2016/12/05 05:43:50.944 kid1| 11,2| client_side.cc(1394) sendStartOfMessage: HTTP Client REPLY:
---------
HTTP/1.1 503 Service Unavailable
Server: squid/3.5.22
Mime-Version: 1.0
Date: Mon, 05 Dec 2016 05:43:50 GMT
Content-Type: text/html;charset=utf-8
Content-Length: 3498
X-Squid-Error: ERR_CONNECT_FAIL 110
Vary: Accept-Language
Content-Language: en
X-Cache: MISS from squid
Connection: close
----------
2016/12/05 05:43:50.944 kid1| 33,2| client_side.cc(817) swanSong: local=*******:3129 remote=############# flags=17
2016/12/05 05:43:50.944 kid1| 20,2| store.cc(980) checkCachable: StoreEntry::checkCachable: NO: not cachable
2016/12/05 05:43:50.944 kid1| 20,2| store.cc(980) checkCachable: StoreEntry::checkCachable: NO: not cachable
</code></pre>
<p>I tried so many different configurations that I'm already lost in
what does work and what doesn't. I'm probably not understanding the
connection between iptables and squid properly, but no matter what I
read I always end up here.</p>
<p>I appreciate any suggestions.</p></div><br></div>