[squid-users] Squid 3.5.2 will only start if cache directory is empty

[Amos Jeffries]

On 20/03/2015 3:26 a.m., Stanford Prescott wrote:
> On Thu, Mar 19, 2015 at 12:11 AM, Amos Jeffries <squid3 at treenet.co.nz>
> wrote:
> 
> 
>> On 19/03/2015 11:29 a.m., Stanford Prescott wrote:
>>> I posted this message to the list a few days ago but haven't received any
>>> responses yet. I am hoping someone might be able to provide some insight
>> on
>>> what is going on.
>>>
>>> I have been trying to get Squid 3.5.2 to work with the Smoothwall Express
>>> 3.1 Linux firewall distribution. Specifically, I have modified the Squid
>>> version included with Smoothwall Express 3.1 to enable HTTPS caching. I
>>> have had this working successfully up to Squid version 3.4.10. Now with
>>> trying to upgrade to Squid 3.5.2 I am having problems that I didn't
>>> encounter with prior versions of Squid.
>>>
>>> The first issue I had, which is now resolved, was improper permissions of
>>> the shm folder (in SWE found in /dev/shm). Changing the folder
>> permissions
>>> to Squid user and group allowed Squid 3.5.2 to start. However, now it
>> will
>>> only start with an empty cache directory.
>>
>> Ouch. /dev/shm is a folder for system shared-memory sockets to be
>> created by applications. It should be owned by root user and group, with
>> 777 permissions. Squid (or the OS kernel) should be able to create
>> "files" inside it, but it should not be owned by Squid.
>>
>>
>>
> When I encountered the "Permission denied" errors for shm_open it was
> suggested by another user that I try changing /dev/shm to squid:root and
> 775. When I made that change the permission denied error went away. I will
> try your suggestion to go back to root:root and permissions of 777.
> 
>>> Once it starts with an empty
>>> cache directory, it seems to function correctly as far as caching SSL
>>> encrypted web pages. However, if Squid needs to be restarted for any
>>> reason, it will not restart until the cache directory
>>> (/var/spool/squid/cache) is emptied.
>>
>> That HTTP data cache is unrelated to the SSL session cache. Its contents
>> should not matter.
>>
>>
>>
> That's what I thought as well but still, squid will only start if I first
> empty that cache directory. Perhaps the patch you suggest below will fix
> this.

Maybe that is a separate side effect of the fast shutdown and crashing
issues. Please try just erasing the swap.state for the cache if it
happens again.


> 
>>> *2015/03/14 00:29:47 kid1| helperOpenServers: Starting 5/5 'ssl_crtd'
>>> processes*
>>> *FATAL: Ipc::Mem::Segment::open failed to
>>> shm_open(/squid-ssl_session_cache.shm): (2) No such file or directory*
>>>
>>
>>>
>>> What is the "squid-ssl_session_cache". Am I supposed to define that
>>> somewhere in the
>>> Squid configuration? Is that why I am getting that error message because
>> an
>>> ssl_session_cache is not defined somewhere?
>>
>> The .shm is the name of a shared memory socket "file" name. You have
>> sslproxy_session_cache_size defined with a size so the SSL session
>> ticket cache is used.
>>
>> Please try patching your Squid with
>> <http://www.squid-cache.org/Versions/v4/changesets/squid-4-13984.patch>.
>> It should resolve many permissions issues Squid 3.5 workers are having
>> on startup.
>>
>> I will try that.
>>>
>>> This is my squid.conf file with SSL caching using ssl-bump enabled.
>>>
>>
>>>
>>> *# A random port for forward-proxy port needed for SSL*
>>> *http_port 8081*
>>>
>>> *http_port 192.168.100.1:800 <http://192.168.100.1:800/> intercept
>> ssl-bump
>>> generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
>>> cert=/var/smoothwall/mods/proxy/ssl_cert/squidCA.pem*
>>>
>>> *https_port 192.168.100.1:808 <http://192.168.100.1:808/> intercept
>>> ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
>>> cert=/var/smoothwall/mods/proxy/ssl_cert/squidCA.pem*
>>
>> Why two ports? one is usually sufficient.
>>
>>
> 
> 
> Without the "http_port 8081" (or any other random port, it seemed) I would
> get "no forward-proxy port" errors when trying to start squid with SSL
> proxying enabled.
> 

Ah sorry. I see my mailers fancy markup bold+italics blinded me to the
missing 's' on the first intercept port. That config was fine (and correct).

> 
>> These were "carry-overs" from the original squid.conf used by earlier
>> versions of Smoothwall. I will remove them.
>>>
>>> *request_header_access Content-Type allow all*
>>> *request_header_access Date allow all*
>>> *request_header_access Host allow all*
>>> *request_header_access If-Modified-Since allow all*
>>> *request_header_access Pragma allow all*
>>> *request_header_access Accept allow all*
>>> *request_header_access Accept-Charset allow all*
>>> *request_header_access Accept-Encoding allow all*
>>> *request_header_access Accept-Language allow all*
>>> *request_header_access Connection allow all*
>>> *request_header_access All allow all*
>>
>> The above settings do nothing but waste CPU time. You can remove them.
>>
>> What you are instructing Squid to do is effectively "allow certain
>> headers X, Y, Z, oh and every other header too".
>>
>>
> These options were suggested to me by a user who wished to be able to put
> Squid in "stealth mode" meaning, for sites that won't accept connections
> from a proxy to mask squid's headers with proxy settings to appear to the
> sites as a non-proxied connection. When the user has a site that requires
> this, the website url is placed as an acl before that block. Should I not
> have those lines unless there is a website that requires them?
> 

Thought so, its not doing what you were told it does. Replace that "All"
entry - which allows every header through explicitly, with:
   request_header_access Other deny all


But there are some more critical HTTP/1.1 headers you will also need to
add allows for to get traffic working properly:
  TE, Transfer-Encoding, Expect, Authorization, Proxy-Authorization,
If-Not-Modified, If-Unmodified-Since, If-Match, If-None-Match, ETag,
Via, Range, If-Range, Content-Encoding, Content-Disposition,

You can remove the Accept, Accept-Language, Accept-Charset headers. They
are optional and often used in browser footprinting.

The Pragma header can also probably go, its only used in requests by
some very old non-HTTP/1.1 software in place of the
Cache-Control:no-cache feature.


>>>
>>> *shutdown_lifetime 3 seconds*
>>
>> NOTE: very short shutdown time can corrupt the HTTP data cache as the
>> memory index does not have enough time to complete saving to disk.
>>
>>
> Another carry-over from earlier versions of Squid. What would be a more
> reasonable time frame for shutdown_lifetime?

We publish Squid with a default of 30sec. Altering from that depends on
how long the client connections tend to be through the proxy. I believe
these days you need it to be long enough to fully download a 1MB object.

Squid starts rejecting new requests as soon as shutdown is required,
then for the shutdown lifetime TTL its just waiting for the existing
transactions to complete. Most clients tend to finish any given download
within 20-30 seconds.

Amos