[squid-dev] What os/cpu platforms do we want to target as a project?

[Alex Rousskov]

On 12/27/21 4:41 AM, Francesco Chemolli wrote:
> On Sun, Dec 26, 2021 at 10:58 PM Alex Rousskov wrote:
>>
>> On 12/26/21 10:30 AM, Francesco Chemolli wrote:
>>> On Sun, Dec 5, 2021 at 10:05 PM Alex Rousskov wrote:
>>>> On 12/5/21 4:44 AM, Francesco Chemolli wrote:
>>>>> I would recommend that we support as main targets:
>>>>> - Linux on x64, arm64, arm32 and, if we can, MIPS
>>>>> - FreeBSD, OpenBSD on x64
>>>>
>>>>> As best-effort:
>>>>> - Windows on x64, with the aim of eventually promoting to primary target
>>>>> - Darwin on x64 and maybe eventually arm64
>>>>> - NetBSD on x64

First of all, thank you for making good progress on this important thread.


> I'm a bit more reluctant with downgrading FreeBSD and (once stable) 
> OpenBSD. They ship Squid as part of their ports, and while the 
> developer community may not be very large, I'd expect the user 
> community to be a bit more prevalent. This would also prevent the
> risk of the project becoming a Linux monoculture.

I share the overall sentiment, but I think all of the above concerns can
continue to be adequately addressed with FreeBSD/OpenBSD being assigned
to the second tier (i.e. the "best-effort" tier you have proposed). IMO,
it makes little sense to name two tiers and then essentially treat the
second named tier as "all other".

The second tier, as proposed, will bring our attention to any
FreeBSD/OpenBSD regressions and, in most cases, we will address them
(because they are usually easy to address). IMO, this level of support
is more than adequate given our lack of resources and relative
unpopularity of those platforms.

Perhaps this difference in approaches is rooted in our current (and
newly discovered!) disagreement on when best-effort/tier-2 results must
be reported (as discussed further below).


> My angle on that is that having arm32 will help us not become a
> 64-bit monoculture, which could bring in regressions in low-level
> data structures. At this stage in the embedded world, as far as I'm
> aware, there are x86-32, arm32, MIPS and RISC-V. Of these arm32 is
> the most prevalent and therefore my preferred choice. It also helps
> that we have a test environment for it already, courtesy of a couple
> of raspberry pi 3.

Same here: I share the sentiment but consider all these arguments
appropriate for tier-2; they are an overkill for tier-1. Squid can
survive without x86-32, arm32, MIPS, and RISC-V support. Yes, we do not
want to accidentally abandon support for those environments, without a
compelling reason, but that is exactly why we are adding tier-2 -- to
eliminate such accidents: Thanks to tier-2 presence, the decision to
regress support for those platforms (in some specific PR scope) will be
deliberate, _not_ accidental.



>>> Once achieved this status on any of these platforms,
>>> regressions on it are release blockers.
>>
>> I do not know what you mean by "release" exactly, but we should not add
>> rules that block schedule-based branching points and major releases[1].
>> [1] https://wiki.squid-cache.org/ReleaseSchedule

Just to avoid misunderstanding, my comments/opinions about minor
releases below apply to the text below. My sentence quoted above is not
about minor releases; it is about major releases; major releases occur
on a fixed schedule; they are not governed by release maintenance rules
that are largely in Amos hands today. Moreover, since no PR can be
merged if it breaks tier-1 support, it is impossible to have a tier-1
regression unless we change the test environment without fixing Squid
first (which we should not do, especially for tier-1 platforms!).


>> As for other/minor releases, I am not against this rule if the
>> maintainer is happy about it.

> Amos, what do you think?

I would like to adjust my comment: I think this additional formal rule
is not needed because we should just let the maintainer decide what
maintenance rules work best (and adjust them as needed), but I am not
against adding that formal rule if Amos wants to add it to his current
set of rules.


>> AFAICT, the primary costs of keeping a platform on a best-effort list
>> are CI testing time and CI maintenance overheads. I suggest capping that
>> best-effort list to keep the total CI test time for one PR commit under,
>> say, 4 hours and granting CI maintenance team a veto on adding (but not
>> removing) platforms. IIRC, we have discussed PR commit testing time a
>> few months ago; if we agreed on another number of hours there, let's
>> take that as a cap.
> 
> The best-effort model supports testing best-effort platforms
> asynchronously, no need to cap there. Regressions would be noted and
> hopefully dealt with in followup PRs. We are already using this
> practice.

The "I will eventually let you know if your old PR broke OpenBSD support
and let you decide whether to revert that PR commit or fix the problem
sometime in the future" approach does not work (and has never worked)
well IMO. We should know about tier-2 regressions _before_ PR is merged
(and excuse them explicitly). For that, tier-2 regressions should be
reported _before_ a PR is merged.

I understand that we may not have the technical capability to achieve
the timely regression reporting goal today, but that should be the goal,
and the tier design/rules should be based on that goal. Otherwise, there
will be little practical difference between best-effort teir-2 and
other/tier-3. Both will occasionally receive post-PR regression reports.
We should eliminate named tier-2 in that case and just let you test
whatever you want, whenever you can, reporting the results whenever you
have the time to do so.

In summary, if we want to define named tiers based, in part, on the PR
reaction to their test results, then those results must be reported
before the reacting PR is merged. Moreover, all other tiers (i.e. tiers
that are not defined in terms of PRs), do not really matter much in
terms of keeping Squid regressions-free. Can we agree to that principle?


Cheers,

Alex.