[squid-users] Internet Browsing very slow after implementing Squid peek & splice + Access log not tracing full URL
Amos Jeffries
squid3 at treenet.co.nz
Wed May 18 12:39:46 UTC 2016
On 18/05/2016 11:05 p.m., Sagar Malve wrote:
> Scenario : I want to block certain HTTPS website using SSL Bump and
> without installing any SSL Certificate on Clients End as I will be
> distributing this Same Network for Mobile Devices so I don't want to keep
> installing certificate in each Mobile Device like Android / IOS / Windows
> etc phones .......
>
> *I have installed Squid 3.5.13 and we have Broadband Connection with speed
Please upgrade. 3.5.19 is now the minimum version to use with SSL-Bump
feature. Due to almost a dozen CVE issues fixed recently most of which
are only bad when SSL-Bump is used.
> 50 Mb/sec. I have gone through lots of document where I found that we can
> Block Https Traffic without installing Certificate by enabling Peek &
> Splice feature.*
>
Lets be accurate:
* You can *terminate* TLS connections with "ssl_bump terminate" rules.
* You cannot send the client errors or rejection notices unless they
trust the Squid cert generators' CA.
>
> *Also I am not getting full URL for HTTPS Traffic in Access Logs ........*
> We have tried to implement Caching DNS Server (Local) but still it didn't
> work then we have given the Google Public DNS .......
>
> Could you please let us know where we are doing mistake .......
>
The URL for HTTPS requests is encrypted. Squid cannot get access to even
see what it is without decrypting the connection. Which is the action
that requires the CA be installed on the clients.
> ------------------- Below is the Configuration file of Squid
> ---------------------------------------
>
> # -------------------------------------
> # Access Control Lists
> # -------------------------------------
> acl localnet src 192.168.0.0/24 # RFC1918 possible internal network
>
> acl SSL_ports port 443
> acl SSL_ports port 8443 # Telecom exclusion
> acl Safe_ports port 80 # http
> acl Safe_ports port 21 # ftp
> acl Safe_ports port 443 # https
> acl Safe_ports port 70 # gopher
> acl Safe_ports port 210 # wais
> acl Safe_ports port 280 # http-mgmt
> acl Safe_ports port 488 # gss-http
> acl Safe_ports port 591 # filemaker
> acl Safe_ports port 777 # multiling http
>
> # Common methods
> acl CONNECT method CONNECT
> acl PURGE method PURGE
> acl GET method GET
>
> # Windows update acls
> acl windowsupdate dstdomain sls.update.microsoft.com.akadns.net
> acl windowsupdate dstdomain windowsupdate.microsoft.com
> acl windowsupdate dstdomain .update.microsoft.com
> acl windowsupdate dstdomain download.windowsupdate.com
> acl windowsupdate dstdomain redir.metaservices.microsoft.com
> acl windowsupdate dstdomain images.metaservices.microsoft.com
> acl windowsupdate dstdomain c.microsoft.com
> acl windowsupdate dstdomain www.download.windowsupdate.com
> acl windowsupdate dstdomain wustat.windows.com
> acl windowsupdate dstdomain crl.microsoft.com
> acl windowsupdate dstdomain sls.microsoft.com
> acl windowsupdate dstdomain productactivation.one.microsoft.com
> acl windowsupdate dstdomain ntservicepack.microsoft.com
>
> # Windows update methods
> acl wuCONNECT dstdomain www.update.microsoft.com
> acl wuCONNECT dstdomain sls.microsoft.com
>
>
> # SSL bump acl
> acl net_bump src "/etc/squid/net.bump"
>
> # TLD acl
> acl block_tld dstdomain "/etc/squid/dstdom.tld"
>
> # -------------------------------------
> # Access parameters
> # -------------------------------------
> # Deny requests to unsafe ports
> http_access deny !Safe_ports
> # Deny CONNECT to other than SSL ports
> http_access deny CONNECT !SSL_ports
>
> # Only allow cachemgr access from localhost
> http_access allow localhost manager
> http_access deny manager
> http_access deny to_localhost
> # Allow purge from localhost
> http_access allow PURGE localhost
> http_access deny PURGE
>
> # Normalize Accept-Encoding to support compression via eCAP
> request_header_access Accept-Encoding deny all
> request_header_replace Accept-Encoding gzip;q=1.0, identity;q=0.5, *;q=0
> # Disable alternate protocols
> request_header_access Alternate-Protocol deny all
> reply_header_access Alternate-Protocol deny all
> # Disable HSTS
> reply_header_access Strict-Transport-Security deny all
> reply_header_replace Strict-Transport-Security max-age=0; includeSubDomains
> # Remove User-Agent from Vary
> reply_header_access Vary deny all
> reply_header_replace Vary Accept-Encoding
> # Workaround 4253
> request_header_access Surrogate-Capability deny all
>
> # Block top level domains
> http_access deny block_tld
> deny_info TCP_RESET block_tld
>
> # Rule allowing access from local networks
> http_access allow localnet
> http_access allow localhost
>
>
> # ICP/HTCP access
> icp_access allow localnet
> icp_access deny all
> htcp_access allow localnet
> htcp_access deny all
>
> # 302 loop
> acl text_mime rep_mime_type text/html text/plain
> acl http302 http_status 302
> store_miss deny text_mime http302
> send_hit deny text_mime http302
>
> # Windows updates rules
> http_access allow CONNECT wuCONNECT localnet
> http_access allow CONNECT wuCONNECT localhost
> http_access allow windowsupdate localnet
> http_access allow windowsupdate localhost
>
> # SSL bump rules
> acl DiscoverSNIHost at_step SslBump1
> # ICQ/MRA must splice first
> acl NoSSLIntercept ssl::server_name_regex -i "/etc/squid/url.nobump"
> ssl_bump splice NoSSLIntercept
> ssl_bump bump net_bump
This will decrypt at step 1 using Squids CA and generated certificate.
*NO* details from the serverHello or the clientHello are available at
that point. The probablility of having problems is very high. The client
getting a TLS warning alert is almost guaranteed.
> acl tls_s3_server_hello at_step SslBump3
>
> # TLS/SSL bumping steps
This is the second time you have a comment like that. This is a sign of
someone copy-n-pasting config snippets from many sources without
understanding what they do.
I also recognise pieces of three config examples myself and Yuri have
given over the past year to different people with *very* different
behaviours they needed out of the proxy.
> ssl_bump peek tls_s1_connect all # peek at the incoming TLS/SSL
> connect data
> ssl_bump splice all # splice the stream:
> pass-through mode
>
Not a good idea to put comments on lines with directives like that. Some
Squid versions and some individual directives do not support it and will
try to check any ACLs matching words in the comment.
BTW, HTTP authentication happening is not a problem in ssl_bump. You can
remove the "all" from that peek line.
> # And finally deny all other access to this proxy
> http_access deny all
>
> # -------------------------------------
> # HTTP parameters
> # -------------------------------------
> # Local Privoxy is cache parent
> cache_peer 127.0.0.1 parent 8118 0 no-query no-digest default
>
> cache_peer_access 127.0.0.1 deny all
WTF? what is the point of configuring a peer that nothing is ever
allowed to go through?
>
> # Don't cache 404 long time
> negative_ttl 5 minutes
> positive_dns_ttl 15 hours
> negative_dns_ttl 1 minutes
Very likely the DNS problems created by altering these from the web
services own DNS TTL is part of your problem.
>
> # -------------------------------------
> # Cache parameters
> # -------------------------------------
> # dhparams is before squid-3.5.12-20151222-r13967
> # tls-dh is AFTER squid-3.5.12-20151222-r13967
> #http_port 3126 ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_certs/squid.crt
> key=/etc/squid/ssl_certs/squid.key options=NO_SSLv3
> tls-dh=/etc/squid/dhparam.pem
> http_port 3127
> http_port 3128 intercept
> # dhparams is before squid-3.5.12-20151222-r13967
> # tls-dh is AFTER squid-3.5.12-20151222-r13967
> https_port 3129 intercept ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_certs/squid.crt
> key=/etc/squid/ssl_certs/squid.key options=NO_SSLv3
> tls-dh=/etc/squid/dhparam.pem
> sslproxy_capath /etc/ssl/certs
> # SINGLE_DH_USE is 3.5 before squid-3.5.12-20151222-r13967
> #sslproxy_options NO_SSLv3,SINGLE_DH_USE
> # SINGLE_ECDH_USE is AFTER squid-3.5.12-20151222-r13967
> sslproxy_options NO_SSLv3,SINGLE_ECDH_USE
> sslproxy_cipher
> EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:HIGH:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS
> sslcrtd_program /usr/lib/squid/ssl_crtd -s /var/lib/squid/ssl_db -M 4MB
>
> # Specify ICP/HTCP explicity
> icp_port 3130
> htcp_port 4827
>
> # Cache manager
> cache_mgr mymail at gmail.com
>
>
>
> # Forces reload-into-ims
> reload_into_ims on
>
> # Hide internal networks details outside
> via off
> forwarded_for delete
>
> # Do not show Squid version
> httpd_suppress_version_string on
>
>
> # Prioritization of local hits
> qos_flows tos local-hit=0x68
>
> # Specify local DNS cache
> dns_nameservers 8.8.8.8
>
This has been demonstrated as a very effective way to make all your
traffic turn into cache MISS' and go very slowly.
The requirements for interception are that the client and the proxy use
the same DNS resolver. This is needed to ensure the *same* results are
seen by both. Google have some fancy logic to detect different sources
of queries and return different results. You can see how this is going
to go wrong?
The best way to operate is to run a recursive resolver of your own. Make
the proxy use it and direct client DNS traffic through it as well.
If you want to Google DNS could be configured as an upstream resolver of
that. Though why you would need to make all your systems rely on access
to Google when you now have a properly functioning recursive resolver of
your own anyway is a puzzle.
> # Cacle squidinternal
> refresh_pattern -i video-srv\.youtube\.squidinternal 0 0% 0
> refresh_pattern -i squidinternal 14400 100% 518400 override-expire
> override-lastmod refresh-ims reload-into-ims ignore-private ignore-auth
> ignore-must-revalidate store-stale ignore-no-store
Using ignore-private and ignore-must-revalidate on the same
refresh_pattern is *extremely* dangerous. Just asking to get your cache
pwned.
Also ignore-auth makes things *not* be cacheable in all the auth related
cases when it would normally be stored by Squid.
> # Keep swf in cache
> refresh_pattern -i \.swf$ 10080 100% 43200 override-expire
> reload-into-ims ignore-private
> # .NET cache
> refresh_pattern -i \.((a|m)s(h|p)x?)$ 10080 100% 43200
> reload-into-ims ignore-private
> # Other long-lived items
> refresh_pattern -i
> \.(jp(e?g|e|2)|gif|png|bmp|ico|svg|web(p|m)|wm(v|a)|flv|f4f|mp(3|4)|ttf|eot|woff2?|(c|x|j)ss|js(t?|px?))(\?.*)?$
> 14400 100% 518400 override-expire override-lastmod reload-into-ims
> ignore-private ignore-no-store ignore-must-revalidate
> refresh_pattern -i
> \.((cs|d?|m?|p?|r?|s?|w?|x?|z?)h?t?m?(l?)|php(3?|5?)|rss|atom|vr(t|ml))(\?.*)?$
> 10080 100% 86400 override-expire override-lastmod reload-into-ims
> ignore-private ignore-no-store ignore-must-revalidate
see above.
> # Default patterns
> refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
> refresh_pattern . 0 20% 4320 reload-into-ims
>
> ------------------------- Squid Configuration End
> ----------------------------------------------
>
> When we give Network Connection Dirtectly through the Router then Internet
> is working fine but when we pass the Network through Squid the Internet
> work very slow .......
Be aware that Squid is doing some, and possibly a lot of extra
processing on each and every packet. Raw speed gain is not one of the
benefits of an intercepting proxy. Which is why accel(erator) mode is
something entirely different.
>
> ---------------- IPTables ---------------------
>
> Chain PREROUTING (policy ACCEPT 25461 packets, 3444K bytes)
> pkts bytes target prot opt in out source
> destination
> 996 55869 DNAT tcp -- eth1 * 0.0.0.0/0
> 0.0.0.0/0 tcp dpt:80 to:192.168.0.200:3128
> 0 0 REDIRECT tcp -- eth0 * 0.0.0.0/0
> 0.0.0.0/0 tcp dpt:80 redir ports 3128
> 3597 211K DNAT tcp -- eth1 * 0.0.0.0/0
> 0.0.0.0/0 tcp dpt:443 to:192.168.0.200:3129
> 0 0 REDIRECT tcp -- eth0 * 0.0.0.0/0
> 0.0.0.0/0 tcp dpt:443 redir ports 3129
NP: your "REDIRECT" rules are not doing anything because you have DNAT
rules that do the same thing(s) to the same packets beforehand.
DNAT or REDIRECT - pick one.
>
> ---------------------------------------------------------------------------------------------------------------------------------
> Access Logs:
> 1463478680.312 33025 192.168.0.66 TCP_TUNNEL/200 3865 CONNECT
> 216.58.199.165:443 - ORIGINAL_DST/216.58.199.165 -
a) A TLS connection to 216.58.199.165:443 was intercepted.
b) It was allowed into the proxy ("http_access allow localnet").
c.1) "splice NoSSLIntercept" matched
OR,
c.2) no SNI was present and "splice all" matched.
What were you expecting?
> 1463478686.862 40 192.168.0.66 TCP_REFRESH_MODIFIED/200 539 GET
> http://kerastasesalonlocator.com/ - ORIGINAL_DST/103.21.58.154 text/html
> 1463478686.880 5 192.168.0.66 TCP_MISS_ABORTED/000 0 GET
> http://kerastasesalonlocator.com/cgi-sys/defaultwebpage.cgi - ORIGINAL_DST/
> 103.21.58.154 -
Client aborted after just 5 milliseconds. Would not seem to be a problem
unless it happens a lot. And probably not a Squid problem even then.
> -----------------------------------------------------------------------------------------------------------------------------------------
>
>
> We have installed Squid on Ubuntu Server 14.04 Ram: 32 GB HDD: 1TB
>
Amos
More information about the squid-users
mailing list