On Sat, Feb 09, 2019 at 06:05:53PM -0500, Todd Zullinger wrote:

SZEDER Gábor wrote:

quoted

Just curious: how did you noticed the missing GPGSM prereq?

I just grep the build output for '# SKIP|skipped:' and then
filter out those which I expect (thing like MINGW and
NATIVE_CRLF that aren't likely to be in a Fedora build).

Far more manual than the slick method you have below. :)

Yeah, but yours show the SKIP cases, too, i.e. when the whole test is
skipped by:

  if check-something
  then
        skip_all="no can do"
        test_done
  fi

I didn't bother with that because in those cases the prereq is not
denoted by a single word, but rather by a human-readable phrase, and
becase 'prove' runs those skipped test scripts at last when running
slowest first, so I could already see them anyway.

quoted

I'm asking because I use a patch for a good couple of months now that
collects the prereqs missed by test cases and prints them at the end
of 'make test'.  Its output looks like this:

  https://travis-ci.org/szeder/git/jobs/490944032#L2358

Since you seem to be interested in that sort of thing as well, perhaps
it would be worth to have something like this in git.git?  It's just
that I have been too wary of potentially annoying other contributors
by adding (what might be perceived as) clutter to their 'make test'
output :)

Indeed, I think that would be useful.  At the very least,
the .missing_prereqs files look quite handy.  I wouldn't
mind the output from 'make test' either, but building
packages surely shifts my perspective toward more verbose
build logs than someone hacking on git regularly and reading
the 'make test' output.

The problem with those files is that a successful 'make test'
automatically and unconditionally removes the whole 'test-results'
directory at the end.  So a separate and optional 'make test ; make
show-missed-prereqs' wouldn't have worked, that's why I did it this
way.

I think it would be better if we kept the 'test-results' directory
even after a successful 'make test', there are some interesting things
to be found there:

  https://public-inbox.org/git/CAM0VKjkVreBKQsvMZ=pEE0NN5gG0MM+XJ0MzCbw1rxi_pR+FXQ@mail.gmail.com/

quoted

quoted

A GIT_TEST_GNUPGHOME_ROOT var to set the root path for the GNUPGHOME
dirs in the tests is one thought I had, but didn't try to put it into
patch form.  Setting the --root test option is probably enough control
for most cases.

A potential issue I see with GIT_TEST_GNUPGHOME_ROOT is that there are
several test scripts involving gpg, and if GIT_TEST_GNUPGHOME_ROOT is
set for the whole 'make test', then they might interfere with each
other when they happen to be run at the same time.

Yeah, I was envisioning that var as something which set the
base dir, under which the normal test directories would
live.  Basically, like setting --root, but only for the
GnuPG bits.

I'm not impressed by that idea (and I'm even less so after
realizing how it would most likely make it harder to gather
up the results in the CI scripts).  I mainly tossed it out
in the hope someone would reply with a better method. ;)

quoted

In the meantime I came up with a '--short-trash-dir' option to
test-lib, which turns 'trash directory.t7612-merge-verify-signatures'
into 'trash dir.t7612'.  It works, but I don't really like it, and it
required various adjustments to the CI build scripts, notably to the
part in 'ci/print-test-failures.sh' that includes the trash dir of
failed test scripts in the build log.

I can certainly live with setting '--root' to a shorter path
and waiting to see if GnuPG upstream will come up with
something a little more friendly to users like us - running
gpg in a test suite.

Are they aware of the issue?

  https://lists.gnupg.org/pipermail/gnupg-users/2017-January/057451.html

suggests to put the socket in '/var/run/user/$(id -u)', but that's for
the "regular" use case, and we should take extra steps to prevent the
tests' gpg from interfering with the gpg of the user running the
tests.  Not sure it would work on macOS.  And ultimately it's not much
different from your GIT_TEST_GNUPGHOME_ROOT suggestion.

Then I stumbled on these patches patches:

  https://lists.zx2c4.com/pipermail/password-store/2017-May/002932.html

suggesting that at least one other project is working around this
issue instead of waiting for upstream to come up with something.

Though if we do just wait it out,
maybe we could/should add a note in t/README or t/lib-gpg.sh
about this to warn others?

Well, a comment could help others to not waste time on figuring out
this "path is too long for a unix domains socket" issue...  but now
they will be able to find this thread in the list archives as well :)



On a related note: did you happen to notice occasional failures with
gpg2 on Fedora builds?  I observed some lately in tests like
'./t7004-tag.sh' or 't7030-verify-tag.sh' on the Travis CI macOS
builds: it appears as if the gpg process were to die mid-verification.
Couldn't make any sense of it yet, though didn't tried particularly
hard either.