[squid-users] Squid 3.5.22 Bug when using Mimetype Detection? rep_mime_type

Tue Jan 3 08:07:57 UTC 2017

On 2017-01-03 07:33, Flashdown wrote:
> Hello together,
> 
> with Squid 3.5.22 I have switched from using a url-regex to Mime Type
> Detection, which seemed to work nicely until now... :/
> 

The thing to be very wary of with this change is that when reply 
blocking the request does still make it through to the server and any 
processing done there still happens.

With things like ticketing systems that means the tickets and any auth 
related token creating has actually happened, even if the client is 
prevented from being told what the created token was.

That is a major difference from URL based blocking, which would reject 
before any server contact.

> OS: Debian Stretch 4.8.0-1-amd64 #1 SMP Debian 4.8.7-1 (2016-11-13)
> x86_64 GNU/Linux
> 

I really advise going to 3.5.23. Several major issues about sending 
wrong replies on certain occasions were fixed there. New .deb should be 
available by now to fix that.

> I faced the following Situation:
> 
> When I globally deny specific mimetypes using a blockfile, then it
> performs as it should, so only mime types I defined in the block file
> are getting blocked, so far so good.
> 
> When I do an exception for a group I belong to like unblocking
> application/octet-stream, then I can download files, so the exception
> works in the first place.
> acl mime_IT rep_mime_type application/octet-stream
> http_reply_access allow IT mime_IT
> 
> Normally internal targets are excluded from the Proxy using Proxy
> Exception lists. But I do not get these settings automatically, so my
> browser did not contain this exception so I was able to discover the
> following behavior:
> 
> The Issue is occuring when browsing to an internal OTRS Web Server via
> FQDN (It's a web ticket system) through the proxy I get "Access
> Denied" from the Proxy on all requests. But when browsing to an online
> OTRS Demo site with the same OTRS version like this one:
> http://itsm-demo.otrs.com/otrs/index.pl then it works. When I now try
> again to access the internal OTRS Server through the proxy it works.
> That's strange, when I now force reload (CTRL+F5 in Firefox) the
> internal OTRS Ticketsystems webpage, I get the "access denied" again.
> 

How is the auth happening between the users and the proxy to find out 
what the groups are?
  The reply checks may be interferring with the auth replies.

> When I remove the exception from the global block list for the group I
> belong to,- here it's IT- then this issue does not occur and the
> website is accessible like it should.
> So I just need to comment out these lines:
> #acl mime_IT rep_mime_type application/octet-stream
> #http_reply_access allow IT mime_IT
> 
> When I add text/html & application/xml to the global block exception,
> then this error does not occur anymore.
> acl mime_IT rep_mime_type application/octet-stream text/html 
> application/xml
> http_reply_access allow IT mime_IT
> 
> So currently I can workaround the issue in 3 different ways:
> 1. Do not create a global mimetype block exception for groups I belong 
> to
> 2. Browse to the start page of an Online Demo OTRS Site and then
> reload the internal Website
> 3. Add text/html & application/xml to my exception even if these
> Mimetypes are not part of the global block list, so they are not
> supposed to be blocked. (I just looked at the internal website and it
> just uses text/html and application/xml on the start page (Login Page)
> so I added them to the exception list for my group and it worked)
> 
> Conclusion: When having a global mime type block and unblocking a
> specific mime type for a specific group, then this group will most
> propably face issues with mime types that are not supposed to be
> blocked. So in case of errors, I need to unblock not blocked mimetypes
> ,too.
> 
> 
> My Squid config for mime type blocking:
> ---------------------------------------
> ## Define Default MIMETYPE ERROR Message and global block access list
> acl block_mimetypes rep_mime_type "/etc/squid/mimetype_blacklist.acl"
> deny_info ERR_BLOCKED_FILES block_mimetypes
> 
> # Configure Execptions
> acl mime_IT rep_mime_type application/octet-stream
> http_reply_access allow IT mime_IT
> 
> acl mime_SpecialGroup rep_mime_type application/octet-stream
> http_reply_access allow SpecialGroup mime_SpecialGroup
> 
> 
> #Applying Global MimeType Block
> http_reply_access deny block_mimetypes

Any other http_reply_access rules? the implicit-default behaviour may be 
having an effect if there are.

> ---------------------------------------
> Squid main config:
> ---------------------------------------
> http_port 8080 ssl-bump generate-host-certificates=on
> dynamic_cert_mem_cache_size=16MB cert=/*********************
> ssl_bump splice localhost
> ssl_bump splice SSL_Exclude

These splice rules may be related to the issue. Any connection that gets 
spliced may be used by other requests without squid being aware of them.

> ssl_bump bump all
> sslproxy_cert_error allow SSL_TrustedSites
> sslproxy_cert_error deny all
> ---------------------------------------
> 
> 
> 
> Contents of mimetype_blacklist.acl:
> ---------------------------------------

That looks okay to me, just remember that it is a regex pattern. So if a 
type entered there matches a sub-string it can block unexpectedly. That 
includes sub-strings in the type parameters.

> 1483381150.095    186 10.3.101.23 TCP_DENIED_REPLY/403 4493 GET
> http://otrs-server.**.**.com/otrs/index.pl - HIER_DIRECT/10.2.1.107
> text/html
> 1483381150.118      3 10.3.101.23 TCP_DENIED_REPLY/403 4451 GET
> http://*****-proxy.**.**.com:8080/squid-internal-static/icons/SN.png -
> HIER_NONE/- text/html

FWIW: the text/html is the type of the HTML error page being delivered 
for the 403 response. The content-type for the original response payload 
which was blocked is not logged.

You can debug with "debug_options 11,2" to get a cache.log trace of what 
messages are happening and what the original server response headers 
were.

Amos