[squid-dev] Porting Squid Debug Log Functionality

[Alex Rousskov]

On 01/27/2017 08:05 AM, Christopher Nighswonger wrote:

> How difficult would it be to abstract Squid's debug log code and port it
> to another application?

It would be relatively easy, but you should not do that IMO because

1. Truly abstracting debugging functionality is actually pretty
difficult while your planned "port it to another application" step makes
that abstraction unnecessary. AFAICT, you do not want to create a
stand-alone debugging library. You just want [better] debugging for
e2guardian.

2. Squid debugging code has many problems, accumulated over the years
and difficult to get rid of in Squid. e2guardian does not have that
baggage and should not pay the price.

My recommendation is to borrow the good parts of Squid _and_ other
applications debugging APIs, adjust them for the e2guardian needs, and
then provide a matching e2guardian implementation.

Here are a few debugging-related aspects you may want to keep in mind:

* The core debug() macro that makes disabled debugging "cheap" from
performance point of view is a pretty standard approach found in many
applications. You should use it. However, do not forget to provide
another interface for doing something (other than pure logging) if
debugging is enabled. This usually means that your API should have a
function that evaluates whether to enable debugging [in a given
context]. The core debug() macro should use that function to guard logging.

* Automatically inserting source code location and function names is a
pretty standard approach found in many applications. You should use it.
However, this automatic annotation should only be done for the
commonly-used "write this debugging info here" part of the debugging
interface. There should be other interfaces for annotation-free logging.

* The idea of non-overlapping hard-coded "debugging sections" is a bad
one. In most cases, those doing debugging want to detail a set of
transactions and/or modules. You should either find a better way to
configure that or just avoid introduction of sectioning until you know
that you need some kind of scope filtering and you know what kind of
scope filtering you need.

* Using debugging verbosity levels is a standard approach that is easy
to abuse. Squid abuses it badly: Human developers are incapable of
correctly selecting one out of ten levels and human users are incapable
of correctly guessing the one out of ten levels they need. The problem
here is similar to the debugging sections discussed in the above bullet.
The two bullets can be merged under a single "scope filtering" headline.

* Standardizing log lines format is an excellent idea that Squid does
not use enough. You cannot or should not standardize the format of
low-level debugging details, but it must be possible to easily extract
individual debugging records and attribute them to the right
process/scope/etc. Until very recently, this was not possible at all in
Squid, and even now there are holes in that area. At the very minimum,
standardize the beginning of every record and use a format that allows
automatic detection of the record end.

* Reducing debugging noise/duplication is important. Squid does a
terrible job here. I am not sure this is an API problem though. Solving
this problem may require establishing rules on what to log when. For
example, do you log the file descriptor or the entry MD5 key with every
debugging record related to that FD/entry? The correct answer is no, but
you have to accompany that easy answer with an approach that allows the
user to easily discover those FD and entry information in a debugging
log that does not repeat them in every record. This problem is partially
related to the "scope filtering" problem discussed earlier.

* Using an API to safely log low-level, possibly binary or specially
formatted data is a good idea. Squid started to use the Raw class for
that, but it needs more bells and whistles such as indentation and MIME
formatting support. Make sure this API can accommodate a growing list of
partially overlapping formatting requirements.

* Supporting concurrent debugging statements is a must for any
non-trivial application because it is very inconvenient and sometimes
impossible to limit debugging expressions to PODs -- many expressions do
call other functions and those other functions may contain their own
debugging statements. Squid added the corresponding support only recently.


The above list is not comprehensive.


HTH,

Alex.