[squid-dev] Strategy about build farm nodes

[Francesco Chemolli]

On Mon, May 17, 2021 at 8:32 PM Alex Rousskov <
rousskov at measurement-factory.com> wrote:

> On 5/17/21 2:17 AM, Francesco Chemolli wrote:
>
> > Our Linux environments are docker containers on amd64, armv7l and arm64.
> > On a roughly monthly cadence, I pull from our dockerfiles repo
> > (https://github.com/kinkie/dockerfiles) and
> > $ make all push
>
> Does that "make push" command automatically switch Jenkins CI to using
> the new/pushed containers? Or is that a separate manual step?
>

Right now that's automatic.
I'm pondering the best way to have a staging step; Docker supports
versioning of images but we're using that feature to implement multiarch.
In order to rely on that I'll need to change the naming conventions we rely
on.


> >>> What I would place on each individual dev is the case where a PR breaks
> >>> something in the trunk-matrix,trunk-arm32-matrix, trunk-arm64-matrix,
> >>> trunk-openbsd-matrix, trunk-freebsd-matrix builds
>
> >> I think you are saying that
> >> some CI tests called trunk-matrix, trunk-arm32-matrix,
> >> trunk-arm64-matrix, trunk-openbsd-matrix, trunk-freebsd-matrix should be
> >> classified as _required_.
>
> > That is how I read the statement too.
>
> ... but it sounds like that is _not_ what you (Francesco) is actually
> proposing because you are essentialy saying "that ideal would be
> impractical" in the paragraph below. Assuming that you are not attacking
> your own proposal, that means Amos and I do not know what your proposal
> is -- we both guessed incorrectly...
>

The overarching objectives are:
- focus on where most users probably are
- allow iterating quickly giving some signal on tracked branches
- allow landing quickly with more signal
- be slow but exhaustive on every other platform we can cover. Do not block
landing new code on less common or harder to test on platforms, but still
rely on devs to own their changes there, too.

> In a word of infinite resources and very efficient testing, sure.
> > But in a space where a single os/compiler combo takes 2hrs on Linux and
> > 4hrs on Freebsd or openbsd, and a full 5-pr-test takes 6 hours end to
> > end, we need to optimize or making any of these requirements blocking
> > would make these times get 4+ times larger (a full trunk-matrix takes
> > just about a day on amd64, 2 days on arm64), and the complaint would
> > then be that development or release is slowed down by the amount of
> > testing done.
>
> FWIW, I think that full round of PR tests (i.e. one initial set plus one
> staging set) should not exceed ~24 hours, _including_ any wait/queuing
> time. This kind of delay should still allow for reasonable progress with
> PRs AFAICT. This includes any release PRs, of course. If we exceed these
> limits (or whatever limits we can agree on), then we should add testing
> resources and/or drop tests.


Is everyone okay with such a slow turnaround?


> > My proposal aims to test/land the PR on the systems where we can be
> > efficient and that are relevant, and fix any remaining after-the-fact
> > issues with followup, PRs, that remain a soft requirement for the dev
> > introducing the change.
>
> Here, I think you are proposing to make some tests optional. Which ones?
>

Not the tests, but the platforms. Things like gentoo, fedora rawhide,
freebsd, openbsd.
Testing is slower there, so results will lag and we need to batch, running
a full test suite every week or so


> > For instance: we currently have a lot of different build-time
> > configurations meant to save core memory in runtime environments. Is it
> > maybe time to revisit this decision and move these checks to run time?
>
> Sure, quality proposals for removing #ifdefs should be welcomed, one
> (group of) #ifdefs at a time. We can warn squid-users in advance in case
> somebody wants to provide evidence of harm.
>

I'm glad this is having buy-in. Are there any other opinions (for or
against)?


> > Unfortunately, one of the problems we have is that we're running blind.
> > We don't know what configurations our users deploy; we can only assume
> > and that makes this conversation much more susceptible to opinions and
> > harder to build consensus on
>
> Yes, we are blind, but I doubt we should care much about actual
> configuration in this specific context. If we can remove an #ifdef
> without serious negative consequences, we should remove it. We can avoid
> long discussions by allowing anybody with concrete evidence of problems
> to block any particular removal. I bet that conservative approach would
> still allow for the removal of many #ifdefs.
>

Sounds good to me

FWIW, I do not think reducing the number of #ifdefs will solve our
> primary CI problems. I believe we should reduce the number of platforms
> we test on and then use Foundation resources to speed up and improve the
> remaining tests.
>

Each test build is right now 4 full builds and test suites:
autodetected, minimal, maximum, everytthing-in-except-for-auth

Can we reduce them to 2? If so, which ones? that would be the guide to
#ifdef removal


> HTH,
>

It does; thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squid-cache.org/pipermail/squid-dev/attachments/20210517/4eb65b93/attachment-0001.htm>