[squid-users] Squid 3.5.22 Bug when using Mimetype Detection? rep_mime_type

Mon Jan 2 18:33:27 UTC 2017

Hello together,

with Squid 3.5.22 I have switched from using a url-regex to Mime Type 
Detection, which seemed to work nicely until now... :/

OS: Debian Stretch 4.8.0-1-amd64 #1 SMP Debian 4.8.7-1 (2016-11-13) 
x86_64 GNU/Linux

I faced the following Situation:

When I globally deny specific mimetypes using a blockfile, then it 
performs as it should, so only mime types I defined in the block file 
are getting blocked, so far so good.

When I do an exception for a group I belong to like unblocking 
application/octet-stream, then I can download files, so the exception 
works in the first place.
acl mime_IT rep_mime_type application/octet-stream
http_reply_access allow IT mime_IT

Normally internal targets are excluded from the Proxy using Proxy 
Exception lists. But I do not get these settings automatically, so my 
browser did not contain this exception so I was able to discover the 
following behavior:

The Issue is occuring when browsing to an internal OTRS Web Server via 
FQDN (It's a web ticket system) through the proxy I get "Access Denied" 
from the Proxy on all requests. But when browsing to an online OTRS Demo 
site with the same OTRS version like this one: 
http://itsm-demo.otrs.com/otrs/index.pl then it works. When I now try 
again to access the internal OTRS Server through the proxy it works. 
That's strange, when I now force reload (CTRL+F5 in Firefox) the 
internal OTRS Ticketsystems webpage, I get the "access denied" again.

When I remove the exception from the global block list for the group I 
belong to,- here it's IT- then this issue does not occur and the website 
is accessible like it should.
So I just need to comment out these lines:
#acl mime_IT rep_mime_type application/octet-stream
#http_reply_access allow IT mime_IT

When I add text/html & application/xml to the global block exception, 
then this error does not occur anymore.
acl mime_IT rep_mime_type application/octet-stream text/html 
application/xml
http_reply_access allow IT mime_IT

So currently I can workaround the issue in 3 different ways:
1. Do not create a global mimetype block exception for groups I belong 
to
2. Browse to the start page of an Online Demo OTRS Site and then reload 
the internal Website
3. Add text/html & application/xml to my exception even if these 
Mimetypes are not part of the global block list, so they are not 
supposed to be blocked. (I just looked at the internal website and it 
just uses text/html and application/xml on the start page (Login Page) 
so I added them to the exception list for my group and it worked)

Conclusion: When having a global mime type block and unblocking a 
specific mime type for a specific group, then this group will most 
propably face issues with mime types that are not supposed to be 
blocked. So in case of errors, I need to unblock not blocked mimetypes 
,too.

My Squid config for mime type blocking:
---------------------------------------
## Define Default MIMETYPE ERROR Message and global block access list
acl block_mimetypes rep_mime_type "/etc/squid/mimetype_blacklist.acl"
deny_info ERR_BLOCKED_FILES block_mimetypes

# Configure Execptions
acl mime_IT rep_mime_type application/octet-stream
http_reply_access allow IT mime_IT

acl mime_SpecialGroup rep_mime_type application/octet-stream
http_reply_access allow SpecialGroup mime_SpecialGroup

#Applying Global MimeType Block
http_reply_access deny block_mimetypes
---------------------------------------
Squid main config:
---------------------------------------
http_port 8080 ssl-bump generate-host-certificates=on 
dynamic_cert_mem_cache_size=16MB cert=/*********************
ssl_bump splice localhost
ssl_bump splice SSL_Exclude
ssl_bump bump all
sslproxy_cert_error allow SSL_TrustedSites
sslproxy_cert_error deny all
---------------------------------------

Contents of mimetype_blacklist.acl:
---------------------------------------
##############
#This is the global blocklist
# Executables: bin exe com dll class
application/x-msdownload
application/octet-stream
application/exe
application/x-exe
application/dos-exe
vms/exe
application/x-winexe
application/msdos-windows
application/x-msdos-program

# .msi
application/x-msi

# .vbs
text/vbscript
text/vbs

# Archives
# .gz
application/gzip
# .z
application/x-compress
# .gtar
application/x-gtar
# .zip
#application/zip
# .tar
application/x-tar
# .rar
application/x-rar-compressed
# .7z
application/x-7z-compressed

# .torrent
application/x-bittorrent
#############
---------------------------------------

Squid Cache: Version 3.5.22
Service Name: squid
Debian linux
configure options:  '--build=x86_64-linux-gnu' '--prefix=/usr' 
'--includedir=${prefix}/include' '--mandir=${prefix}/share/man' 
'--infodir=${prefix}/share/info' '--sysconfdir=/etc' 
'--localstatedir=/var' '--libexecdir=${prefix}/lib/squid3' '--srcdir=.' 
'--disable-maintainer-mode' '--disable-dependency-tracking' 
'--disable-silent-rules' 'BUILDCXXFLAGS=-g -O2 
-fdebug-prefix-map=/usr/src/mycompile/squid3-3.5.22=. -fPIE 
-fstack-protector-strong -Wformat -Werror=format-security -Wdate-time 
-D_FORTIFY_SOURCE=2 -fPIE -pie -Wl,-z,relro -Wl,-z,now -Wl,--as-needed' 
'--datadir=/usr/share/squid' '--sysconfdir=/etc/squid' 
'--libexecdir=/usr/lib/squid' '--mandir=/usr/share/man' 
'--enable-inline' '--disable-arch-native' '--enable-async-io=8' 
'--enable-storeio=ufs,aufs,diskd,rock' 
'--enable-removal-policies=lru,heap' '--enable-delay-pools' 
'--enable-cache-digests' '--enable-icap-client' 
'--enable-follow-x-forwarded-for' 
'--enable-auth-basic=DB,fake,getpwnam,LDAP,NCSA,NIS,PAM,POP3,RADIUS,SASL,SMB' 
'--enable-auth-digest=file,LDAP' 
'--enable-auth-negotiate=kerberos,wrapper' 
'--enable-auth-ntlm=fake,smb_lm' 
'--enable-external-acl-helpers=file_userip,kerberos_ldap_group,LDAP_group,session,SQL_session,time_quota,unix_group,wbinfo_group' 
'--enable-url-rewrite-helpers=fake' '--enable-eui' '--enable-esi' 
'--enable-icmp' '--enable-zph-qos' '--enable-ecap' 
'--disable-translation' '--with-swapdir=/var/spool/squid' 
'--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid.pid' 
'--with-filedescriptors=65536' '--with-large-files' '--enable-ssl' 
'--enable-ssl-crtd' '--with-openssl' '--with-default-user=proxy' 
'--enable-build-info=Debian linux' '--enable-linux-netfilter' 
'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 
-fdebug-prefix-map=/usr/src/mycompile/squid3-3.5.22=. -fPIE 
-fstack-protector-strong -Wformat -Werror=format-security -Wall' 
'LDFLAGS=-fPIE -pie -Wl,-z,relro -Wl,-z,now -Wl,--as-needed' 
'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' 'CXXFLAGS=-g -O2 
-fdebug-prefix-map=/usr/src/mycompile/squid3-3.5.22=. -fPIE 
-fstack-protector-strong -Wformat -Werror=format-security'

1483381150.095    186 10.3.101.23 TCP_DENIED_REPLY/403 4493 GET 
http://otrs-server.**.**.com/otrs/index.pl - HIER_DIRECT/10.2.1.107 
text/html
1483381150.118      3 10.3.101.23 TCP_DENIED_REPLY/403 4451 GET 
http://*****-proxy.**.**.com:8080/squid-internal-static/icons/SN.png - 
HIER_NONE/- text/html

-- 
Best regards,
Enrico Heine

This email and any files transmitted with it are confidential and 
intended solely for the use of the individual or entity to whom they are 
addressed. If you have received this email in error please notify the 
system manager. This message contains confidential information and is 
intended only for the individual named. If you are not the named 
addressee you should not disseminate, distribute or copy this e-mail. 
Please notify the sender immediately by e-mail if you have received this 
e-mail by mistake and delete this e-mail from your system. If you are 
not the intended recipient you are notified that disclosing, copying, 
distributing or taking any action in reliance on the contents of this 
information is strictly prohibited.