[squid-dev] Strategy about build farm nodes

[Alex Rousskov]

On 5/17/21 2:17 AM, Francesco Chemolli wrote:

> Our Linux environments are docker containers on amd64, armv7l and arm64.
> On a roughly monthly cadence, I pull from our dockerfiles repo
> (https://github.com/kinkie/dockerfiles) and
> $ make all push

Does that "make push" command automatically switch Jenkins CI to using
the new/pushed containers? Or is that a separate manual step?


>>> What I would place on each individual dev is the case where a PR breaks
>>> something in the trunk-matrix,trunk-arm32-matrix, trunk-arm64-matrix,
>>> trunk-openbsd-matrix, trunk-freebsd-matrix builds

>> I think you are saying that
>> some CI tests called trunk-matrix, trunk-arm32-matrix,
>> trunk-arm64-matrix, trunk-openbsd-matrix, trunk-freebsd-matrix should be
>> classified as _required_.

> That is how I read the statement too.

... but it sounds like that is _not_ what you (Francesco) is actually
proposing because you are essentialy saying "that ideal would be
impractical" in the paragraph below. Assuming that you are not attacking
your own proposal, that means Amos and I do not know what your proposal
is -- we both guessed incorrectly...


> In a word of infinite resources and very efficient testing, sure.
> But in a space where a single os/compiler combo takes 2hrs on Linux and
> 4hrs on Freebsd or openbsd, and a full 5-pr-test takes 6 hours end to
> end, we need to optimize or making any of these requirements blocking
> would make these times get 4+ times larger (a full trunk-matrix takes
> just about a day on amd64, 2 days on arm64), and the complaint would
> then be that development or release is slowed down by the amount of
> testing done.

FWIW, I think that full round of PR tests (i.e. one initial set plus one
staging set) should not exceed ~24 hours, _including_ any wait/queuing
time. This kind of delay should still allow for reasonable progress with
PRs AFAICT. This includes any release PRs, of course. If we exceed these
limits (or whatever limits we can agree on), then we should add testing
resources and/or drop tests.


> My proposal aims to test/land the PR on the systems where we can be
> efficient and that are relevant, and fix any remaining after-the-fact
> issues with followup, PRs, that remain a soft requirement for the dev
> introducing the change.

Here, I think you are proposing to make some tests optional. Which ones?


> For instance: we currently have a lot of different build-time
> configurations meant to save core memory in runtime environments. Is it
> maybe time to revisit this decision and move these checks to run time?

Sure, quality proposals for removing #ifdefs should be welcomed, one
(group of) #ifdefs at a time. We can warn squid-users in advance in case
somebody wants to provide evidence of harm.


> Unfortunately, one of the problems we have is that we're running blind.
> We don't know what configurations our users deploy; we can only assume
> and that makes this conversation much more susceptible to opinions and
> harder to build consensus on

Yes, we are blind, but I doubt we should care much about actual
configuration in this specific context. If we can remove an #ifdef
without serious negative consequences, we should remove it. We can avoid
long discussions by allowing anybody with concrete evidence of problems
to block any particular removal. I bet that conservative approach would
still allow for the removal of many #ifdefs.


FWIW, I do not think reducing the number of #ifdefs will solve our
primary CI problems. I believe we should reduce the number of platforms
we test on and then use Foundation resources to speed up and improve the
remaining tests.


HTH,

Alex.