[squid-dev] [RFC] The situation with helpers/

[Alex Rousskov]

On 01/13/2016 06:12 AM, Kinkie wrote:
> On Tue, Jan 5, 2016 at 6:01 PM, Alex Rousskov
> <rousskov at measurement-factory.com> wrote:
>> On 01/04/2016 08:58 PM, Amos Jeffries wrote:
>>> The SBuf and CharacterSet patch auditing has brought up the questions of
>>> why helpers/ is separate from src/, built first, what cross-dependency
>>> there can be, and even why we are bundling C++ helpers to begin with.
>>>
>>> So lets clarify it formally once and for all.
>>>
>>> Q1: why is helpers/ built before src/ ?
>>
>> A1: Primarily for historical and paranoia reasons.
>>
>>
>>> Q2: why is helpers/ separate from src/ ?
>>
>> A2: Primarily for historical and paranoia reasons.
>>
>>
>>> should we shuffle them back to src/*/helpers/ directories on a
>>> per-component basis ?
>>
>> Yes w.r.t. src/. There is no known good reason to enact artificial hard
>> barriers between parts of Squid code. All Squid code should be in src/.

> Could or could not. For sure it would help with dependency tracking
> where needed, and there are cases where it is needed.

Yes, that is one reason I said "should", not "could".


>>> * it does somewhat hinder anyone wanting to manually build a single
>>> helper. Though I have not heard of anyone doing so in many years.
>>
>> The move does not preclude anyone from building a helper. The fact that
>> a few more things may be built along the way is a minor detail IMO.
>> Patches to improve dependency tracking (by migrating away from recursive
>> Makefiles??) are welcomed.
> 
> Ugh. see src/Makefile.am for the monster file this would yield.

And see src/Makefile.am for the monster file the current
lets-track-complex-dependencies-manually approach has created. I do not
know exactly what the best solution to the current src/Makefile.am
problem is. I know that is a big problem. I suspect the solution
involves more than simply migrating away from recursive Makefiles. I do
not want to discuss that problem in this thread. That is why I said
exactly what I said, including the question marks.


>>> * it does mean we should probably discard the goal of having the helpers
>>> be demo code examples for people to base their custom helpers on.

>> The location of the helper code is pretty much irrelevant for the above
>> goal. However, if we want to provide demo helpers that are easy to
>> understand and customize, then we probably should not use C++ to write
>> those demo helpers.

> I don't agree. We should use whatever tool makes sense, depending on
> the availability of libraries, complexity, volunteer time etc.

Not sure what you do not agree with:

* Your statement does not contradict what I said about the location of
the helper code being pretty much irrelevant.

* If you are focusing on the second part of my statement, then please
note that it is possible, perhaps even likely that the "tools" that
"make sense" in our situation are not appropriate for some goals. When
that happens, the goals have to be re-evaluated to match reality so that
unrealistic goals do not misdirect our decisions.


>> * If we want to provide the smallest set of default helpers for the
>> widest possible set of deployment environments, then internal
>> dependencies are preferred to the external ones and, hence, C++ is the
>> best language choice.
>>
>> * If we want to provide default production-quality helpers for the most
>> demanding deployment environments, then internal dependencies are
>> irrelevant, and C++ is the best language choice. Please note that I do
>> not know whether it is wise to declare performance the primary goal,
>> given the performance deficiencies of the helper interface itself and
>> the current Squid Project resources.
>> Any open source helper can be examined, customized, and cloned, of
>> course. Our decision with regard to programming language and
>> dependencies may make it easier or more difficult, but not impossible.
>> It is a question of the primary optimization target: ease of
>> understanding/customizability versus supported platforms versus performance.
> 
> I believe our goals should be, in descending order of importance:
> 1- cover a reasonable set of needs
> 2- provide a gallery of examples for administrators who have special
> needs not covered by the provided set of helpers. 

Goal #2, as stated, is not a goal but a side-effect: Whatever helpers we
provide to satisfy goal #1 will automatically become a "gallery of
examples".


> Corollaries of this objective are:
> 2a- bundled helpers should aim at being as readable as possible.

Any code should aim at being as readable as possible so this statement
is not a corollary and does not really bring anything new to the debate.


> 2b- bundled helpers should try to avoid relying on Squid internal code
> within reason. We can expect administrators embarking in the task to
> be competent programmers, asking them to be also competent Squid
> coders is maybe too much to ask.

Relying on Squid libraries/APIs is not that much different than relying
on external libraries/APIs. Thus, I disagree that a reasonable use of
Squid code in helpers forces helper writers to become competent Squid
coders. It is a question of how much use is "reasonable" given our
still-being-debated goals, of course.


Alex.