Re: [PATCH] ci: update freebsd 12 cirrus job

On Fri, Aug 13, 2021 at 2:39 AM SZEDER Gábor [off-list ref] wrote:

On Thu, Aug 12, 2021 at 01:22:00PM -0700, Carlo Marcelo Arenas Belón wrote:

quoted

diff --git a/.cirrus.yml b/.cirrus.yml
index c2f5fe385a..e114ffee1a 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml

@@ -2,8 +2,15 @@ env:
   CIRRUS_CLONE_DEPTH: 1

 freebsd_12_task:
+  env:
+    GIT_PROVE_OPTS: "--timer --jobs 10"

Why these prove options?

that line came from the Github Actions part of ci/lib.sh.

--jobs is the one I was going after, but kept --timer for consistency.

On other CI systems we pass 'prove' the option
'--state=failed,slow,save' as well to reduce runtime.  However, this
only works when there is a persistent place for prove's state files,
e.g. the cache feature of Travis CI.

note that when GitHub Actions was added[1], it was explicitly excluded
because it wasn't really that effective

If Cirrus CI lacks a similar
feature, then we can't benefit from this option, but it'd be worth
mentioning in the commit message.

They do have the option and will likely be a small change that I could
send as a follow up, but wasn't included in this change since it was
"controversial" as pointed above.

note also that the other uses of it aren't really being exercised,
since AFAIK neither azure or travis are running now, and indeed last
time I tried to run CI in my own account it didn't end up well, with
no successful run and running out of credit before I could even
attempt to fix them all.

indeed, was even going to suggest (after release) that they are
removed, which will also make the code cleaner for integrating cirrus
(which still has a usable opensource plan).

quoted

+    GIT_TEST_OPTS: "--no-chain-lint --no-bin-wrappers"

Why these test options?

again lifted from the "GitHub Actions" code just for consistency and
to avoid breaking if either of those are broken (considering the only
other running CI does test with those unconditionally), but I realize
now that I might have been too conservative since my focus with this
commit was "speed and accuracy".

chain-linting is done by a mighty sed script; I think it's worth
running it with FreeBSD's 'sed' as well.

indeed, even if I recall correctly it was Solaris sed the one that
broke, and I would expect anyone breaking this chainlinting will
notice very quickly on their local run.

Quoting 't/README', '--no-bin-wrappers' "can speed up test runs
especially on platforms where running shell scripts is expensive".  Is
running shell scripts really that expensive on FreeBSD?

no, I am assuming they were added with Windows in mind, but note that
as configure they also affect all the linux builds through GitHub
Actions.
Both these flags might be worth adding back for all non Windows
systems, but doing so will be easier when using the "ci" scripts for
all environments and after it has been discussed IMHO.

OTOH, why are there no options that would show us some information
about test failures, i.e. why no '--verbose-log -x --immediate' like
on other CI systems?

I wasn't the original author, so can't answer why it wasn't done; but
I sure think the next step will be to add support for it to the ci/
scripts and with that make it more consistent, which likely will
include that change.

This patch was just the minimal change to make sure that it runs
efficiently and is representative of current users.

Carlo

[1] https://lore.kernel.org/git/40fe4f7e2c34c6997a6824ef0dc6aad7a71a4434.1586538752.git.congdanhqx@gmail.com/ (local)