From: Jonathan Nieder <hidden> Date: 2021-05-20 20:41:33
Hi,
Siavash wrote:
[Subject: [Bug] git-credential-netrc.perl is not built and is not available in `exec-path`]
which is located in `contrib/credential/netrc`.
I'm missing some basic information:
- what platform are you running on?
- what commands did you use to install Git?
- what are you trying to do? That can help us with understanding the
problem or giving better advice tailored to the use case. The netrc
credential helper is mostly meant as a demo, since libcurl supports
~/.netrc on its own already.
Thanks and hope that helps,
Jonathan
Things in contrib are not built by default because they don't
necessarily work everywhere. For example, the osxkeychain credential
helper won't compile on Linux because the requisite shared libraries
are specific to macOS. You'll need to compile them manually and install
them in a suitable location.
Note that that location can be someplace like ~/bin, if that's in your
PATH. For example, since the Debian packages don't yet ship the
libsecret credential helper, I've built it and placed it there. Now
that I've done that, git help -a shows git credential-libsecret as an
option.
--
brian m. carlson (he/him or they/them)
Houston, Texas, US
Things in contrib are not built by default because they don't
necessarily work everywhere. For example, the osxkeychain credential
helper won't compile on Linux because the requisite shared libraries
are specific to macOS. You'll need to compile them manually and install
them in a suitable location.
I agree with this, but just following up with a bit of a devil's
advocate: why not put osxkeychain into a regular "make install", but
make it conditional via a Makefile knob, like we do for other
platform-specific features?
The big reason most helpers are in contrib/ is because I wanted them to
be true third-party contributions that do not share any code with Git.
That avoids licensing questions when linking with libraries, and puts
them on an equal footing for people who want to implement a helper for
their favorite obscure tool.
It has lead to a somewhat funny situation, though. I don't really
consider osxkeychain well maintained. It's not built nor tested as part
our regular build. I wrote it long ago, but I never actually _used_ it
day-to-day, as I've never had a mac. It doesn't seem to have gotten any
commits since 2013.
And yet, my impression is that basically every Git user on macOS is
using it every day, because both Apple Git and homebrew build it by
default (and I think at least in the case of Apple Git, it's hard-coded
into the config). A little scary, but nobody seems to have complained. :)
I wonder if we could build it and run it through t0303 as part of the
mac CI process (though I recall at the time that it was really finicky
for automated testing; it wouldn't even run over an ssh session).
Likewise, we probably could be building and testing the libsecret ones
via the Linux CI job (I don't use those either myself, but presumably
they pass t0303).
Note that that location can be someplace like ~/bin, if that's in your
PATH. For example, since the Debian packages don't yet ship the
libsecret credential helper, I've built it and placed it there. Now
that I've done that, git help -a shows git credential-libsecret as an
option.
One curiosity is that:
cd contrib/credential/netrc
make install
builds and installs into .../libexec/git-core. And we don't seem to
include that in "git help -a" (it's not listed in our generated command
list, but nor is it an "external command" found in the $PATH).
-Peff
That part is OK. The netrc build runs "make SCRIPT_PERL=...", which will
override any instances where it is set in the top-level Makefile.
So just doing:
cd contrib/credential/netrc
make install
should be enough to make it work.
-Peff
Hi,
On 2021-05-21 01:11:26 +0430, Jonathan Nieder [off-list ref] wrote:
I'm missing some basic information:
- what platform are you running on?
I'm using Arch Linux with Linux v5.12.5-arch1-1.
- what commands did you use to install Git?
`sudo pacman -S git`
- what are you trying to do? That can help us with understanding the
problem or giving better advice tailored to the use case. The netrc
credential helper is mostly meant as a demo, since libcurl supports
~/.netrc on its own already.
I wanted to use `git-credential-netrc.perl` to use `.authinfo.gpg` as
git credential. The `gitcredentials` manual said use:
`git config --global credential-helper foo`
for using credential helpers. It also mentioned that `git credential-`
gets prepended to the external helpers. So I expected:
`git config --global credential-helper 'netrc'`
to work.
I can use it as a helper if I use an absolute path:
'/usr/share/git/credential/netrc/git-credential-netrc.perl'
After looking at the `PKGBUILD` file in Arch Linux's git repository, I
found out that they are not making "netrc".
https://github.com/archlinux/svntogit-packages/blob/4864e39da0bc99e373f3cb728272a93d66b58cd6/trunk/PKGBUILD#L63-L64
So this is not a bug. As Jeff King mentioned as well.
I should have checked the `PKGBUILD`.
The output of `git --exec-path` is `/usr/lib/git-core`. And there are
other helpers there(made by Arch Linux).
This is the output of `lsd /usr/lib/git-core/ | rg credential`:
git-credential
git-credential-cache
git-credential-cache--daemon
git-credential-gnome-keyring
git-credential-libsecret
git-credential-store
I expected the third-party helpers to be in the output of `git help -a`
as mentioned in the manual. They are not in the `PATH`, but they are in
git's `exec-path`. So shouldn't they be printed in the help output?
From: brian m. carlson <hidden> Date: 2021-05-21 22:27:44
On 2021-05-21 at 10:04:53, Jeff King wrote:
On Fri, May 21, 2021 at 01:55:32AM +0000, brian m. carlson wrote:
quoted
Things in contrib are not built by default because they don't
necessarily work everywhere. For example, the osxkeychain credential
helper won't compile on Linux because the requisite shared libraries
are specific to macOS. You'll need to compile them manually and install
them in a suitable location.
I agree with this, but just following up with a bit of a devil's
advocate: why not put osxkeychain into a regular "make install", but
make it conditional via a Makefile knob, like we do for other
platform-specific features?
Sure, let's do it.
For osxkeychain, it's probably pretty simple to always build it, since
macOS will always have the appropriate libraries if the compiler is
installed. I would be in favor of also building by default on Linux and
having a Makefile knob to disable that, since the requisite libraries are
a part of nearly every distribution and doing so will spur distros to
ship it, which many do not.
And yet, my impression is that basically every Git user on macOS is
using it every day, because both Apple Git and homebrew build it by
default (and I think at least in the case of Apple Git, it's hard-coded
into the config). A little scary, but nobody seems to have complained. :)
I wonder if we could build it and run it through t0303 as part of the
mac CI process (though I recall at the time that it was really finicky
for automated testing; it wouldn't even run over an ssh session).
Likewise, we probably could be building and testing the libsecret ones
via the Linux CI job (I don't use those either myself, but presumably
they pass t0303).
Running the tests will be harder. macOS, I believe, requires an
interactive session to have the keychain unlocked, and on Linux, you
require gnome-keyring or an equivalent daemon running, which practically
means that you need a desktop session.
--
brian m. carlson (he/him or they/them)
Houston, Texas, US
From: Jeff King <hidden> Date: 2021-05-23 20:07:50
On Fri, May 21, 2021 at 10:27:08PM +0000, brian m. carlson wrote:
quoted
I agree with this, but just following up with a bit of a devil's
advocate: why not put osxkeychain into a regular "make install", but
make it conditional via a Makefile knob, like we do for other
platform-specific features?
Sure, let's do it.
For osxkeychain, it's probably pretty simple to always build it, since
macOS will always have the appropriate libraries if the compiler is
installed.
Hmm. So I tried just building oxskeychain in our CI via contrib, and it
fails. :(
It looks like it's OK with clang, but not gcc:
https://github.com/peff/git/runs/2647748209?check_suite_focus=true
Maybe nobody cares about gcc for this use, but I'm inclined to leave it
to somebody who actually runs macOS to poke at further.
I would be in favor of also building by default on Linux and
having a Makefile knob to disable that, since the requisite libraries are
a part of nearly every distribution and doing so will spur distros to
ship it, which many do not.
I assume you mean contrib/credential/libsecret here. I'd worry that
flipping it on by default is annoying for people who build from source.
I don't have dev libraries for things like libsecret or glib on my
development system[1], and I'd be surprised if most server-oriented
machines have even the non-dev versions.
Which isn't an argument against making it easier to build them from the
main Makefile, but it seems like having them on by default (with a
NO_LIBSECRET) would cause more surprises than the other way around
(USE_LIBSECRET or similar).
I suspect that just opening a bug report against distro packages might
get some traction (especially if it comes with a patch to create the
extra package). I do wonder if packagers are hesitant to reach into
contrib/, not knowing how well maintained the contents are (to be
honest, I am not confident in how well maintained they are, either; it
might be helpful if somebody who routinely used each helper stepped up
to say that they would maintain it).
-Peff
[1] I was actually surprised I had libsecret installed at all.
Apparently pinentry-gtk2 links against it, which I have for use with
gpg. But my system might not be considered typical anyway, as I
don't use any kind of "desktop environment".
From: Felipe Contreras <hidden> Date: 2021-05-24 03:01:26
Jeff King wrote:
I suspect that just opening a bug report against distro packages might
get some traction (especially if it comes with a patch to create the
extra package).
I have tried that; do doesn't work. If git developers have $x in
"contrib" it's for a reason.
I do wonder if packagers are hesitant to reach into
contrib/,
Of course they are! The word "contrib" has an obvious meaning.
This is precisely the reason why I tried to graduate "contrib/completion"
out of "contrib" to no avail [1].
If you want to install anything from "contrib" by default, it should be
"contrib/completion" first, which is already very well tested, and
shipped by many distributions. I would move it outside "contrib", to be
semantically consistent.
Then maybe "contrib/credential", sure, following "contrib/completion".
But to do a mish-mash between "/" and "/contrib". Does not make sense.
You can't eat your cake and have it too.
Either they are contributed, or they aren't.
Cheers.
[1] https://lore.kernel.org/git/1399607587-14127-1-git-send-email-felipe.contreras@gmail.com/
[1] I was actually surprised I had libsecret installed at all.
Apparently pinentry-gtk2 links against it, which I have for use with
gpg. But my system might not be considered typical anyway, as I
don't use any kind of "desktop environment".
Unless you have a backend (like gnome-keyring) libsecret is useless.
Can you do "secret-tool lookup *" and fetch any of your passwords?
Cheers.
--
Felipe Contreras
I suspect that just opening a bug report against distro packages might
get some traction (especially if it comes with a patch to create the
extra package).
I have tried that; do doesn't work. If git developers have $x in
"contrib" it's for a reason.
quoted
I do wonder if packagers are hesitant to reach into
contrib/,
Of course they are! The word "contrib" has an obvious meaning.
[Minor edit to the quoted text to inline the link]:
Seems like that patch just got no replies at the time. FWIW I'd very
much be for it and would encourage you to re-submit it.
I'm not sure s/shared/contrib/g is the best naming though, but maybe I'm
contributing to needless bikeshedding by mentioning that.
You apparently named it like that to match where distros usually install
it (/usr/share), but we also have docs there, locale, and the perl/
directory usually (well, at least on my distro) ends up there.
I wonder if just a top-level completion/* wouldn't be best, or if we
want to group them all together something like
optional/{completion,credential}/ or other name suggesting that these
are meant to interact with not-always-present 3rd party software. Maybe
integrations/* ?
For some of these names a general re-arrangement of contrib/* would be a
logical thing to follow, e.g. I think it would make sense to carve out
various ci/, contrib/coccinelle, Documentation/doc-diff,
check-builtins.sh etc. and other "only for supporting git.git
development" or "only called by our own Makefile(s)" into some
consistently named pattern.
I'm also very much in favor of building and testing all of this software
by default, to the best of our ability. We've had some avoidable bitrot
e.g. in subtree and mw-to-git in the past, some of that is a pain to
test (e.g. requiring an installed MediaWiki), but we can usually
build/test some part of it (e.g. in that case, does it even compile as
Perl code?). In other cases we could compile/test things by default on
certain platforms if they're platform-specific.
From: Felipe Contreras <hidden> Date: 2021-05-24 17:21:59
Ævar Arnfjörð Bjarmason wrote:
On Sun, May 23 2021, Felipe Contreras wrote:
quoted
Jeff King wrote:
quoted
I suspect that just opening a bug report against distro packages might
get some traction (especially if it comes with a patch to create the
extra package).
I have tried that; do doesn't work. If git developers have $x in
"contrib" it's for a reason.
quoted
I do wonder if packagers are hesitant to reach into
contrib/,
Of course they are! The word "contrib" has an obvious meaning.
[Minor edit to the quoted text to inline the link]:
Seems like that patch just got no replies at the time. FWIW I'd very
much be for it and would encourage you to re-submit it.
I'm not sure s/shared/contrib/g is the best naming though, but maybe I'm
contributing to needless bikeshedding by mentioning that.
It is the best location because that's where completions go.
You can check the location bash-completion suggests to install
completions to:
% pkg-config --variable=completionsdir bash-completion
/usr/share/bash-completion/completions
In the case of zsh it's /usr/share/zsh/site-functions.
Additionally, if you install them in your home directory, it should be
$XDG_DATA_HOME/bash-completion/completions.
$XDG_DATA_HOME is $HOME/.local/share (analogous to /usr/share).
You apparently named it like that to match where distros usually install
it (/usr/share), but we also have docs there, locale, and the perl/
directory usually (well, at least on my distro) ends up there.
Distributions install them there, because that's where they are expected
(by bash-completion and zsh).
I wonder if just a top-level completion/* wouldn't be best, or if we
want to group them all together something like
optional/{completion,credential}/ or other name suggesting that these
are meant to interact with not-always-present 3rd party software. Maybe
integrations/* ?
extra/ is a better name.
However, there's already many things that are optional, like gitk and
git-gui, do they belong there too? For that matter locales are optional
too.
I think if such a decison to have an extra/ directory is made, it should
be orthogonal to the completion graduation.
For some of these names a general re-arrangement of contrib/* would be a
logical thing to follow, e.g. I think it would make sense to carve out
various ci/, contrib/coccinelle, Documentation/doc-diff,
check-builtins.sh etc. and other "only for supporting git.git
development" or "only called by our own Makefile(s)" into some
consistently named pattern.
Me too.
I'm also very much in favor of building and testing all of this software
by default, to the best of our ability. We've had some avoidable bitrot
e.g. in subtree and mw-to-git in the past, some of that is a pain to
test (e.g. requiring an installed MediaWiki), but we can usually
build/test some part of it (e.g. in that case, does it even compile as
Perl code?). In other cases we could compile/test things by default on
certain platforms if they're platform-specific.
Yeah, some spring cleaning would be good.
I'll put sending the patch in my unending to-do list.
Cheers.
--
Felipe Contreras
I suspect that just opening a bug report against distro packages might
get some traction (especially if it comes with a patch to create the
extra package).
I have tried that; do doesn't work. If git developers have $x in
"contrib" it's for a reason.
quoted
I do wonder if packagers are hesitant to reach into
contrib/,
Of course they are! The word "contrib" has an obvious meaning.
[Minor edit to the quoted text to inline the link]:
Seems like that patch just got no replies at the time. FWIW I'd very
much be for it and would encourage you to re-submit it.
I'm not sure s/shared/contrib/g is the best naming though, but maybe I'm
contributing to needless bikeshedding by mentioning that.
It is the best location because that's where completions go.
You can check the location bash-completion suggests to install
completions to:
% pkg-config --variable=completionsdir bash-completion
/usr/share/bash-completion/completions
In the case of zsh it's /usr/share/zsh/site-functions.
Additionally, if you install them in your home directory, it should be
$XDG_DATA_HOME/bash-completion/completions.
$XDG_DATA_HOME is $HOME/.local/share (analogous to /usr/share).
*Nod* I mean just because it ends up there in the FHS doesn't mean it's
best for us to mirror that structure in git.git.
quoted
You apparently named it like that to match where distros usually install
it (/usr/share), but we also have docs there, locale, and the perl/
directory usually (well, at least on my distro) ends up there.
Distributions install them there, because that's where they are expected
(by bash-completion and zsh).
quoted
I wonder if just a top-level completion/* wouldn't be best, or if we
want to group them all together something like
optional/{completion,credential}/ or other name suggesting that these
are meant to interact with not-always-present 3rd party software. Maybe
integrations/* ?
extra/ is a better name.
However, there's already many things that are optional, like gitk and
git-gui, do they belong there too? For that matter locales are optional
too.
I think if such a decison to have an extra/ directory is made, it should
be orthogonal to the completion graduation.
The line I was attempting to draw was components that optionally
interact with optional 3rd party software.
The i18n framework isn't like that because we build it and interact with
ourselves, ditto for say PCRE. Optional, but /usr/bin/git is using it.
As opposed to bash/zsh completions, git will run just fine without
either of those shells installed.
The git-gui and gitk programs are also first-party software, just like
git-send-email or whatever. We just have knobs not to build them because
of the dependencies. It looks like we might be spinning them away from
git.git entirely in slow-motion, but so far they're first-class
commands.
quoted
For some of these names a general re-arrangement of contrib/* would be a
logical thing to follow, e.g. I think it would make sense to carve out
various ci/, contrib/coccinelle, Documentation/doc-diff,
check-builtins.sh etc. and other "only for supporting git.git
development" or "only called by our own Makefile(s)" into some
consistently named pattern.
Me too.
quoted
I'm also very much in favor of building and testing all of this software
by default, to the best of our ability. We've had some avoidable bitrot
e.g. in subtree and mw-to-git in the past, some of that is a pain to
test (e.g. requiring an installed MediaWiki), but we can usually
build/test some part of it (e.g. in that case, does it even compile as
Perl code?). In other cases we could compile/test things by default on
certain platforms if they're platform-specific.
Yeah, some spring cleaning would be good.
I'll put sending the patch in my unending to-do list.
From: Felipe Contreras <hidden> Date: 2021-05-25 01:24:08
Ævar Arnfjörð Bjarmason wrote:
On Mon, May 24 2021, Felipe Contreras wrote:
quoted
Ævar Arnfjörð Bjarmason wrote:
quoted
quoted
I'm not sure s/shared/contrib/g is the best naming though, but maybe I'm
contributing to needless bikeshedding by mentioning that.
It is the best location because that's where completions go.
You can check the location bash-completion suggests to install
completions to:
% pkg-config --variable=completionsdir bash-completion
/usr/share/bash-completion/completions
In the case of zsh it's /usr/share/zsh/site-functions.
Additionally, if you install them in your home directory, it should be
$XDG_DATA_HOME/bash-completion/completions.
$XDG_DATA_HOME is $HOME/.local/share (analogous to /usr/share).
*Nod* I mean just because it ends up there in the FHS doesn't mean it's
best for us to mirror that structure in git.git.
It's not just that it ends there, it's how it ends there.
Right now the Arch Linux's git package does this:
find contrib/ -name '.gitignore' -delete
cp -a ./contrib/* "$pkgdir"/usr/share/git/
I would rather have an install-shared target to populate /usr/share/git.
Having a standar location for distributions would allow scripts to
simplify instructions, like:
source /usr/share/git/completion/prompt.sh
Sure, how install-shared populates /usr/share/git is kind of orthogonal,
but it would make sense for install-shared to install stuff from shared/.
quoted
quoted
You apparently named it like that to match where distros usually install
it (/usr/share), but we also have docs there, locale, and the perl/
directory usually (well, at least on my distro) ends up there.
Distributions install them there, because that's where they are expected
(by bash-completion and zsh).
quoted
I wonder if just a top-level completion/* wouldn't be best, or if we
want to group them all together something like
optional/{completion,credential}/ or other name suggesting that these
are meant to interact with not-always-present 3rd party software. Maybe
integrations/* ?
extra/ is a better name.
However, there's already many things that are optional, like gitk and
git-gui, do they belong there too? For that matter locales are optional
too.
I think if such a decison to have an extra/ directory is made, it should
be orthogonal to the completion graduation.
The line I was attempting to draw was components that optionally
interact with optional 3rd party software.
The i18n framework isn't like that because we build it and interact with
ourselves, ditto for say PCRE. Optional, but /usr/bin/git is using it.
As opposed to bash/zsh completions, git will run just fine without
either of those shells installed.
git will also run fine without git-send-email, git-instaweb, and git-p4.
The git-gui and gitk programs are also first-party software, just like
git-send-email or whatever. We just have knobs not to build them because
of the dependencies. It looks like we might be spinning them away from
git.git entirely in slow-motion, but so far they're first-class
commands.
I know what is the status quo, but when talking about suggestions for
improvement the status quo does not matter. Either the status quo makes
sense, or it doesn't.
I know gitk is "first-class", but *should* it? If so, why?
And I know git-completion.bash isn't "first-class", but shouldn't it? If
so, why not?
We even run git-completion.bash tests by default, gitk doesn't have
tests. We don't even track its history; all the commits are squashed
into a single "merge". And surely bash is a much more likely dependency
to be present in the user's system than Tcl/Tk.
So I don't really see what gitk has, that git-completion.bash hasn't.
Either both belong in extra/, or none of them do (and if one does an the
other doesn't, to me it's clear which is which).
Cheers.
--
Felipe Contreras