[squid-users] SSL Bump Failures with Google and Wikipedia

Sat Sep 30 21:18:39 UTC 2017

Hey Jeffrey,

What happens when you disable the next icap service this way:
icap_service service_avi_resp respmod_precache icap://127.0.0.1:1344/cherokee bypass=0
adaptation_access service_avi_resp deny all

Is it still the same?
What I suspect is that the requests are defined to accept gzip compressed objects and the icap service is not "gnuzip" them which results in what you see.

To make sure that squid is not at fault here try to disable both icap services and then add then one at a time and see which of this triangle is giving you trouble.
I enhanced an ICAP library which is written in GoLang at:
https://github.com/elico/icap

And I have couple examples on how to work with http requests and responses at:
https://github.com/andybalholm/redwood/
https://github.com/andybalholm/redwood/search?utf8=%E2%9C%93&q=gzip&type=

Let me know if you need help finding out the issue.

All The Bests,
Eliezer

----
Eliezer Croitoru
Linux System Administrator
Mobile: +972-5-28704261
Email: eliezer at ngtech.co.il

-----Original Message-----
From: squid-users [mailto:squid-users-bounces at lists.squid-cache.org] On Behalf Of Jeffrey Merkey
Sent: Saturday, September 30, 2017 23:28
To: squid-users <squid-users at lists.squid-cache.org>
Subject: [squid-users] SSL Bump Failures with Google and Wikipedia

Hello All,

I have been working with the squid server and icap and I have been
running into problems with content cached from google and wikipedia.
Some sites using https, such as Centos.org work perfectly with ssl
bumping and I get the decrypted content as html and it's readable.
Other sites, such as google and wikipedia return what looks like
encrypted traffic, or perhaps mime encoded data, I am not sure which.

Are there cases where squid will default to direct mode and not
decrypt the traffic?  I am using the latest squid server 3.5.27.  I
really would like to get this working with google and wikipedia.  I
reviewed the page source code from the browser viewer and it looks
nothing like the data I am getting via the icap server.

Any assistance would be greatly appreciated.

The config I am using is:

#
# Recommended minimum configuration:
#

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed

acl localnet src 127.0.0.1
acl localnet src 10.0.0.0/8     # RFC1918 possible internal network
acl localnet src 172.16.0.0/12  # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7       # RFC 4193 local private network range
acl localnet src fe80::/10      # RFC 4291 link-local (directly
plugged) machines

acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
acl CONNECT method CONNECT

#
# Recommended minimum Access Permission configuration:
#
# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# Only allow cachemgr access from localhost
http_access allow localhost manager
http_access deny manager

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#

# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost

# And finally deny all other access to this proxy
http_access deny all

# Squid normally listens to port 3128
#http_port 3128

# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /usr/local/squid/var/cache/squid 100 16 256

# Leave coredumps in the first cache dir
coredump_dir /usr/local/squid/var/cache/squid

#
# Add any of your own refresh_pattern entries above these.
#
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern .               0       20%     4320

http_port 3128 ssl-bump generate-host-certificates=on
dynamic_cert_mem_cache_size=4MB cert=/etc/squid/ssl_cert/myCA.pem
http_port 3129

# SSL Bump Config
always_direct allow all
ssl_bump server-first all
sslproxy_cert_error deny all
sslproxy_flags DONT_VERIFY_PEER
sslcrtd_program /usr/local/squid/libexec/ssl_crtd -s /var/lib/ssl_db
-M 4MB sslcrtd_children 8 startup=1 idle=1

# For squid 3.5.x
#sslcrtd_program /usr/local/squid/libexec/ssl_crtd -s /var/lib/ssl_db -M 4MB

# For squid 4.x
# sslcrtd_program /usr/local/squid/libexec/security_file_certgen -s
/var/lib/ssl_db -M 4MB

icap_enable on
icap_send_client_ip on
icap_send_client_username on
icap_client_username_header X-Authenticated-User
icap_preview_enable on
icap_preview_size 1024
icap_service service_avi_req reqmod_precache icap://127.0.0.1:1344/request bypass=1
adaptation_access service_avi_req allow all

icap_service service_avi_resp respmod_precache icap://127.0.0.1:1344/cherokee bypass=0
adaptation_access service_avi_resp allow all

Jeff
_______________________________________________
squid-users mailing list
squid-users at lists.squid-cache.org
http://lists.squid-cache.org/listinfo/squid-users