[squid-users] Survey on assertions: When the impossible happens

[Alex Rousskov]

On 02/27/2016 05:02 AM, joe wrote:
> isn't better to use for  kid1| assertion failed: store.cc:1890: "isEmpty()" 
> in store.cc line 1890 assert (isEmpty());
> insted replace it with VERIFY (isEmpty());

> assert   force the program to halt that's why the restart happen
> and its very bad wen squid restart doing huge rebuild it take time to
> function normal
> assert is good for development so isn't better to replace it  ????


The big question is what to replace it with... Replacing every
assert(foo) with code that explicitly handles both "foo" and "not foo"
cases is like writing down all possible chess games. Impractical!

I would like to use your question to highlight an ongoing discussion
that may affect all Squid admins in the foreseeable future. Please see
below for the choices we are discussing and for specific questions you
can help us with.


* Introduction.

Asserts trigger when Squid faces an impossible situation. In this
context, "impossible" means impossible from the code/developers point of
view. Clearly, that point of view was wrong, but it helps to think of
this as an "impossible" situation because it reflects the resulting
chaos when an impossible becomes possible.

For example, imagine you assert that "traveling back in time is
impossible" and frame your society norms and laws around that assertion.
Then, suddenly, the assertion triggers and time travel becomes a
reality. How long do you think "carrying on as usual" will work after
that assertion is violated?

Most Squid assertions are not as fundamental as the laws of physics, of
course, but many are pretty much equivalent as far as the surrounding
code is concerned.


* Choices.

Overall, there are three options for handling an impossible situation:

1. Quit Squid process. This is what Squid does today in most cases.
   When the impossible happens, you get a crash. Very predictable.
   No malformed/corrupted/misleading HTTP messages (some are truncated).
   No memory leaks.

2. Quit the current processing sequence but keep Squid process running,
   assuming that [most] other processing sequences are not affected.
   [If you are familiar with programming, this is done by throwing
   exceptions instead of asserting and catching those exceptions at
   "processing sequence" boundaries].

3. Keep executing the current processing sequence, assuming that the
   assertion was wrong or unimportant. This is what you might be
   suggesting above. When the impossible happens, you may get a crash,
   memory leaks, malformed/corrupted/misleading HTTP messages, or normal
   behavior, depending on the assertion and traffic.

IMO, we should make #2 the default, but make the choice between all
three options configurable by the admin (without recompiling Squid).

Please note that I did not detail #2 side effects. They are somewhere
between ~100% clarity of option #1 and ~100% chaos of option #3. With
option #2, you _will_ get all of the bad side effects of option #3, but,
hopefully, with significantly lower frequency, especially long-term.

Why so much uncertainty about #2? Current Squid code does not identify
and isolate many processing sequences well enough and does not handle
their abandonment well. In other words, it is a code quality matter. We
are working on improving it, but it will take time and resources. Today,
some sequences can be terminated nicely, but terminating others will
result in unpredictable effects similar to option #3.

This is why making #2 the default now is controversial.


* Feedback.

There is an ongoing discussion about the best approach to handling
impossible situations among active Squid developers. Constructive
feedback from a Squid admin point of view is welcomed!

Q0: Do you think an admin should be able to control the choice among the
three options detailed above?

Q1: Which option is the best default?

Q2: Would you prefer to see fewer assertions in exchange for more memory
leaks and an increased probability of malformed/corrupted/misleading
HTTP messages?


Thank you,

Alex.