[squid-dev] Squid-4 status update

[Alex Rousskov]

On 03/26/2017 09:20 PM, Amos Jeffries wrote:

> Below are the bugs which are currently preventing a "stable" release:


> 4659 - sslproxy_foreign_intermediate_certs does not work any more
> 
>  * waiting for someone to identify what part of the Downloader code
> breaks this directive behaviour.

Factory will work on this one if nobody beats us to it. We should be
able to start (and provide an ETA) within a week or so.


> Below are planned to ignore for release purposes. If they can be fixed
> great, but I'm not holding for them:

IMO, it is your call whether to violate the formal release conditions as
long as you acknowledge and explain the violations. Those formal
conditions allow us to knowingly release completely buggy software so it
is difficult to defend them in general.


> 4662 - build errors with LibreSSL 2.4.4
> 
>  * stuck waiting for someone to add feature detection to all the code
> using OPENSSL_VERSION macros. Then to find any other places LibreSSL
> differ and fix those too.

FWIW, the above nice long-term solution is not required to close that
bug IMO. A much smaller short-term solution would be to introduce a
version-testing macro that will auto-correct LibreSSL lies about the
supported OpenSSL API version.


> this kind of requires someone who actually use LibreSSL to do the
> testing.

Anybody can test with LibreSSL, but somebody who needs LibreSSL support
may want to perform or motivate the corresponding work. I agree that the
bug is stuck for now.


> 4505 - Memory hits shadow disk entries and vice versa
> 
>  * AFAIK, still nobody working on this. Requires a fair bit of redesign
> in store code.

Factory is working on this major bug. The fix was stuck in my review
queue, but I should be able to start the review within a few days. I
hope to see the final patch posted here soon after the PID file fixes
(discussed below).


> 4631 - security_file_certgen drops request due to a full queue
> 
>  * seems to be an expected but undesirable consequence of being helper
> API. The effects are major, but the fix is technically a feature
> enhancement AFAICT.

My understanding of that bug report is completely different: The current
working theory is that some kind of unusual input or condition (e.g., a
very large certificate or malformed helper request) results in more and
more helpers getting independently stuck waiting for more input from
Squid that never comes.

The bug will remain stuck until somebody volunteers to analyze the
latest logs from Dan (at least).


> Any other issues that dont have bug reports I should wait for?

I recall several issues that might be worth waiting for (your call):

1. PID file management changes

We fixed some of the problems in trunk r13867 but the current code still
badly mishandles SMP race conditions. We see a stream of related problem
reports from SMP installations that cannot reliably start, restart, or
send signals. The fix went through several major rewrites and review
cycles already, so I hope we are within a week of the final solution.

The fix contains some Squid "interface" changes (exit codes, what
failures are considered fatal, level-1 messages, etc.) so it may be a
good idea to get it in before a lot of folks start upgrading to v4. It
is your call though.


2. New transaction_initiator ACL

Based on squid-users and private requests, quite a few admins are likely
to need this ACL to better cope with regression-like problems related to
other recent improvements. Here is a quote from the being-reviewed patch
preamble:

> This ACL is essential in several use cases, including:
> 
> * After fetching a missing intermediate certificate, Squid uses the
>   regular cache (and regular caching rules) to store the response. Squid
>   deployments that do not want to cache regular traffic need to cache
>   fetched certificates and only them.
> 
>   acl fetched_certificate transaction_initiator certificate-fetching
>   cache allow fetched_certificate
>   cache deny all
> 
> * Many traffic policies and tools assume the existence of an HTTP client
>   behind every transaction. Internal Squid requests violate that
>   assumption. Identifying internal requests protects external ACLs, log
>   analyzers, and other mechanisms from the transactions they mishandle.
> 
>   acl skip_logging transaction_initiator internal
>   access_log ... !skip_logging

I do not know whether v4 port is practical but it would be nice to have
this ACL in v4.


3. Bug 4682 - With ssl_bump, squid can ignore ACL denied and serve page

This is a v3 bug, but it is almost fixed, and since the bug may have
disastrous consequences, you may want to get it in before calling v4
"stable".


4. [PATCH] Do not forward HTTP requests to dead idle peers

Eduard has posted that improvement yesterday. If you think this is a bug
fix, then it can be committed to v4 later. Otherwise, its reviewed/final
version should be committed to v4 before calling v4 stable IMO.


BTW, please do not rush to create any new official branches after
changing v4 status: During to-git migration, I expect us to have enough
trouble with v5 already being "out of place", and creating more such
branches before conversion is likely to make the matters worse.


Thank you,

Alex.