Re: [PATCH] ci: update freebsd 12 cirrus job

Hi Gábor,

On Fri, 13 Aug 2021, SZEDER Gábor wrote:

On Thu, Aug 12, 2021 at 01:22:00PM -0700, Carlo Marcelo Arenas Belón wrote:

quoted

make sure it uses a supported OS branch and uses all the resources
that can be allocated efficiently.

while only 1GB of memory is needed, 2GB is the minimum for a 2 CPU
machine (the default), but by increasing parallelism wall time has
been reduced by 35%.

Signed-off-by: Carlo Marcelo Arenas Belón <redacted>
---
 .cirrus.yml | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index c2f5fe385a..e114ffee1a 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml

@@ -2,8 +2,15 @@ env:
   CIRRUS_CLONE_DEPTH: 1

 freebsd_12_task:
+  env:
+    GIT_PROVE_OPTS: "--timer --jobs 10"

Why these prove options?

The only problem I see with it is that its `--jobs 10` disagrees with the
`MAKEFLAGS: -j4` below. Either both should say 10, or both should say 4.

On other CI systems we pass 'prove' the option
'--state=failed,slow,save' as well to reduce runtime.  However, this
only works when there is a persistent place for prove's state files,
e.g. the cache feature of Travis CI.  If Cirrus CI lacks a similar
feature, then we can't benefit from this option, but it'd be worth
mentioning in the commit message.

We also don't benefit from this in the GitHub workflow because there is
likewise no easy way to persist the state.

quoted

+    GIT_TEST_OPTS: "--no-chain-lint --no-bin-wrappers"

Why these test options?

I guess that's for the same reason we use these options in the Windows
tests: it speeds up things for a historically pretty slow build axis. At
least I seem to see FreeBSD runs lagging behind all the time.

chain-linting is done by a mighty sed script; I think it's worth
running it with FreeBSD's 'sed' as well.

Quoting 't/README', '--no-bin-wrappers' "can speed up test runs
especially on platforms where running shell scripts is expensive".  Is
running shell scripts really that expensive on FreeBSD?

I don't think it is FreeBSD per se, but the available VMs that make this
speed-up worth our while.

OTOH, why are there no options that would show us some information
about test failures, i.e. why no '--verbose-log -x --immediate' like
on other CI systems?

That's because we don't use the `ci/` scripts at all, and the failures are
not actually shown.

This is a mild annoyance to me, I have to admit: there is very little in
the way of actionable information whenever the FreeBSD jobs fail.

Lucky for me, I've so far only encountered breakages that _also_ affected
the macOS build axes, and therefore I could diagnose the issues in the
GitHub workflow run logs.

But yes, it would be nice if the FreeBSD CI runs were more helpful in case
of problems.

As far as this here patch goes: I am in favor of integrating it as-is.

Ciao,
Dscho

quoted

+    MAKEFLAGS: "-j4"
+    DEFAULT_TEST_TARGET: prove
+    DEVELOPER: 1
   freebsd_instance:
-    image: freebsd-12-1-release-amd64
+    image_family: freebsd-12-2
+    memory: 2G
   install_script:
     pkg install -y gettext gmake perl5
   create_user_script:
--
2.33.0.rc1.379.g2890ef5eb6