From: Jason Pyeron <hidden> Date: 2021-01-22 20:42:00
From: Jason Pyeron <redacted>
Sent: Friday, January 22, 2021 3:09 PM
I am about to make a release for logwatch tonight. Historically the files are owned by logwatch in the
tgz file. When I use git archive it is owned by uid 0, is there an option to set the uid/uname,
gid/gname owner of the files?
Answer: not at this time, as it is hard coded in the source.
archive-tar.c:
static void prepare_header(struct archiver_args *args,
struct ustar_header *header,
unsigned int mode, unsigned long size)
{
xsnprintf(header->mode, sizeof(header->mode), "%07o", mode & 07777);
xsnprintf(header->size, sizeof(header->size), "%011"PRIoMAX , S_ISREG(mode) ? (uintmax_t)size : (uintmax_t)0);
xsnprintf(header->mtime, sizeof(header->mtime), "%011lo", (unsigned long) args->time);
xsnprintf(header->uid, sizeof(header->uid), "%07o", 0);
xsnprintf(header->gid, sizeof(header->gid), "%07o", 0);
strlcpy(header->uname, "root", sizeof(header->uname));
strlcpy(header->gname, "root", sizeof(header->gname));
meh.
From: René Scharfe <hidden> Date: 2021-01-22 21:04:51
Am 22.01.21 um 21:40 schrieb Jason Pyeron:
quoted
From: Jason Pyeron <redacted>
Sent: Friday, January 22, 2021 3:09 PM
I am about to make a release for logwatch tonight. Historically the files are owned by logwatch in the
tgz file. When I use git archive it is owned by uid 0, is there an option to set the uid/uname,
gid/gname owner of the files?
Answer: not at this time, as it is hard coded in the source.
archive-tar.c:
static void prepare_header(struct archiver_args *args,
struct ustar_header *header,
unsigned int mode, unsigned long size)
{
xsnprintf(header->mode, sizeof(header->mode), "%07o", mode & 07777);
xsnprintf(header->size, sizeof(header->size), "%011"PRIoMAX , S_ISREG(mode) ? (uintmax_t)size : (uintmax_t)0);
xsnprintf(header->mtime, sizeof(header->mtime), "%011lo", (unsigned long) args->time);
xsnprintf(header->uid, sizeof(header->uid), "%07o", 0);
xsnprintf(header->gid, sizeof(header->gid), "%07o", 0);
strlcpy(header->uname, "root", sizeof(header->uname));
strlcpy(header->gname, "root", sizeof(header->gname));
meh.
Adding support for using a custom user and group should be easy. Is
this just a cosmetic thing? Regular users would ignore the user info in
the archive, and root should not be used for extracting, and on systems
that don't have a logwatch user this wouldn't make a difference anyway,
right?
René
From: Jason Pyeron <hidden> Date: 2021-01-22 21:15:17
From: René Scharfe
Sent: Friday, January 22, 2021 4:00 PM
Am 22.01.21 um 21:40 schrieb Jason Pyeron:
quoted
quoted
From: Jason Pyeron
Sent: Friday, January 22, 2021 3:09 PM
I am about to make a release for logwatch tonight. Historically the files are owned by logwatch in
the
quoted
quoted
tgz file. When I use git archive it is owned by uid 0, is there an option to set the uid/uname,
gid/gname owner of the files?
Answer: not at this time, as it is hard coded in the source.
archive-tar.c:
static void prepare_header(struct archiver_args *args,
struct ustar_header *header,
unsigned int mode, unsigned long size)
{
xsnprintf(header->mode, sizeof(header->mode), "%07o", mode & 07777);
xsnprintf(header->size, sizeof(header->size), "%011"PRIoMAX , S_ISREG(mode) ?
Adding support for using a custom user and group should be easy. Is
this just a cosmetic thing? Regular users would ignore the user info in
In this case, likely a cosmetic thing.
the archive, and root should not be used for extracting, and on systems
But I can think of situations, where it would matter - and I do not agree that "root should not be used for extraction".
that don't have a logwatch user this wouldn't make a difference anyway,
right?
I updated the hard coded values to match what was needed, did a make and presto all was happy.
The backend should take:
--group=NAME
--group-id=GID
--owner=NAME
--owner-id=UID
Patch wanted?
From: Konstantin Ryabitsev <hidden> Date: 2021-01-22 21:41:03
On Fri, Jan 22, 2021 at 10:00:04PM +0100, René Scharfe wrote:
Adding support for using a custom user and group should be easy. Is
this just a cosmetic thing? Regular users would ignore the user info in
the archive, and root should not be used for extracting, and on systems
that don't have a logwatch user this wouldn't make a difference anyway,
right?
Right now, "git archive" operations are bit-for-bit identical across all
versions going back at least 8+ years. In fact, we've been relying on this to
support bundling tarball signatures with git tags themselves (via git notes).
E.g. you can see this in action here:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tag/?h=v5.10.9
If you click on "(sig)", you will download a signature that can be used to
verify the tarball generated using "git archive".
I would argue that adding user/group support to "git archive" operation is
not really solving any problems other than "it's different from when I run it
as a regular user" -- and can introduce potential compatibility problems if
implemented.
So, I would selfishly vote not to implement this.
-K
From: Jason Pyeron <hidden> Date: 2021-01-22 22:15:27
From: Konstantin Ryabitsev
Sent: Friday, January 22, 2021 4:40 PM
Subject: Re: git archive setting user and group
On Fri, Jan 22, 2021 at 10:00:04PM +0100, René Scharfe wrote:
quoted
Adding support for using a custom user and group should be easy. Is
this just a cosmetic thing? Regular users would ignore the user info in
the archive, and root should not be used for extracting, and on systems
that don't have a logwatch user this wouldn't make a difference anyway,
right?
Right now, "git archive" operations are bit-for-bit identical across all
versions going back at least 8+ years. In fact, we've been relying on this to
support bundling tarball signatures with git tags themselves (via git notes).
E.g. you can see this in action here:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tag/?h=v5.10.9
If you click on "(sig)", you will download a signature that can be used to
verify the tarball generated using "git archive".
I would argue that adding user/group support to "git archive" operation is
not really solving any problems other than "it's different from when I run it
as a regular user" -- and can introduce potential compatibility problems if
Being pedantic here, it is different than when I run it as any user - including root.
Don’t confuse tar x with tar c.
tar c captures the current owner of the files, or allows override with a single user / map file.
implemented.
So, I would selfishly vote not to implement this.
-K
On Fri, Jan 22, 2021 at 10:00:04PM +0100, René Scharfe wrote:
quoted
Adding support for using a custom user and group should be easy. Is
this just a cosmetic thing? Regular users would ignore the user info in
the archive, and root should not be used for extracting, and on systems
that don't have a logwatch user this wouldn't make a difference anyway,
right?
Right now, "git archive" operations are bit-for-bit identical across all
versions going back at least 8+ years. In fact, we've been relying on this to
support bundling tarball signatures with git tags themselves (via git notes).
E.g. you can see this in action here:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tag/?h=v5.10.9
If you click on "(sig)", you will download a signature that can be used to
verify the tarball generated using "git archive".
I would argue that adding user/group support to "git archive" operation is
not really solving any problems other than "it's different from when I run it
as a regular user" -- and can introduce potential compatibility problems if
implemented.
So, I would selfishly vote not to implement this.
It seems "logwatch" has the same situation, except their backwards
compatibility is with non-"git archive" tool that used user != root.
If it solves a problem for some users and someone comes up with a patch
I don't see why it shouldn't be implemented, but I think it's important
that "root" is the default unless configured or the relevant option is
invoked.
Or do you mean that the kernel.org use-case is that users are expected
to run "git archive" on their own machines and upload the result to
kernel.org, and that kernel.org relies on two such files bit-for-bit
identical?
From: brian m. carlson <hidden> Date: 2021-01-23 01:06:12
On 2021-01-22 at 21:39:54, Konstantin Ryabitsev wrote:
On Fri, Jan 22, 2021 at 10:00:04PM +0100, René Scharfe wrote:
quoted
Adding support for using a custom user and group should be easy. Is
this just a cosmetic thing? Regular users would ignore the user info in
the archive, and root should not be used for extracting, and on systems
that don't have a logwatch user this wouldn't make a difference anyway,
right?
Right now, "git archive" operations are bit-for-bit identical across all
versions going back at least 8+ years. In fact, we've been relying on this to
support bundling tarball signatures with git tags themselves (via git notes).
E.g. you can see this in action here:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tag/?h=v5.10.9
If you click on "(sig)", you will download a signature that can be used to
verify the tarball generated using "git archive".
Please do not rely on this behavior. I want to state in the strongest
possible terms that this is not guaranteed behavior and it may change at
any time. We have explicitly said so on the list multiple times. If
you need reproducible archives, you need to add a tool to canonicalize
them in a suitable format and not rely on Git to never change things.
If you are relying on this behavior right now, I urge you to change that
at your earliest possible convenience. I don't want to break
kernel.org's infrastructure again, but I'm also not going to tiptoe
around sending patches in fear of that, nor feel bad if it happens again
for this reason.
I would argue that adding user/group support to "git archive" operation is
not really solving any problems other than "it's different from when I run it
as a regular user" -- and can introduce potential compatibility problems if
implemented.
I agree that this feature isn't really something we want. Git produces
tar archives for software interchange, in which case producing an
intentionally anonymous tarball is the desired behavior.
--
brian m. carlson (he/him or they/them)
Houston, Texas, US
From: Jeff King <hidden> Date: 2021-01-23 04:59:07
On Sat, Jan 23, 2021 at 01:05:00AM +0000, brian m. carlson wrote:
quoted
Right now, "git archive" operations are bit-for-bit identical across all
versions going back at least 8+ years. In fact, we've been relying on this to
support bundling tarball signatures with git tags themselves (via git notes).
E.g. you can see this in action here:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tag/?h=v5.10.9
If you click on "(sig)", you will download a signature that can be used to
verify the tarball generated using "git archive".
Please do not rely on this behavior. I want to state in the strongest
possible terms that this is not guaranteed behavior and it may change at
any time. We have explicitly said so on the list multiple times. If
you need reproducible archives, you need to add a tool to canonicalize
them in a suitable format and not rely on Git to never change things.
I strongly second this. :)
It's also not quite true that things have remained bit-for-bit identical
for all that time. We have fixed bugs in that time, although they do not
always cause a change in every output tarball (they often depend on
corner cases like having long pathnames). Two off the top of my head
(that have indeed caused people to complain about changing checksums):
- 22f0dcd963 (archive-tar: split long paths more carefully,
2013-01-05)
- 82a46af13e (archive-tar: fix pax extended header length calculation,
2019-08-17)
We also rely on system gzip. That's pretty stable, but I have heard tell
that even `gzip -n` may differ on platforms.
Another fun one I saw recently: using export-subst with $Format:%h$ will
produce different results depending on how many objects are present in
the repository running git-archive.
-Peff
From: Konstantin Ryabitsev <hidden> Date: 2021-01-23 05:13:46
On Sat, Jan 23, 2021 at 01:05:00AM +0000, brian m. carlson wrote:
quoted
Right now, "git archive" operations are bit-for-bit identical across all
versions going back at least 8+ years. In fact, we've been relying on this to
support bundling tarball signatures with git tags themselves (via git notes).
E.g. you can see this in action here:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/tag/?h=v5.10.9
If you click on "(sig)", you will download a signature that can be used to
verify the tarball generated using "git archive".
Please do not rely on this behavior. I want to state in the strongest
possible terms that this is not guaranteed behavior and it may change at
any time. We have explicitly said so on the list multiple times. If
you need reproducible archives, you need to add a tool to canonicalize
them in a suitable format and not rely on Git to never change things.
It doesn't need to be perpetually the same. This is the reason the comments
mention git version -- "to generate an archive that matches this signature,
run this command (with this git version)." I know there is no guarantee that
this won't ever change across versions, but I don't see why we can't expect
git-archive to return the same bit-for-bit output when using the same version
of git. Adding --owner and --group would potentially change that, especially
if that's something settable via git-config.
If you are relying on this behavior right now, I urge you to change that
at your earliest possible convenience. I don't want to break
kernel.org's infrastructure again, but I'm also not going to tiptoe
around sending patches in fear of that, nor feel bad if it happens again
for this reason.
It won't break, it will just inconvenience folks who are using this perk --
Greg KH will just have to use the same git version that we have on the server
when he generates the signatures. This has happened before.
-K
From: Konstantin Ryabitsev <hidden> Date: 2021-01-23 05:17:38
On Fri, Jan 22, 2021 at 11:58:19PM -0500, Jeff King wrote:
We also rely on system gzip. That's pretty stable, but I have heard tell
that even `gzip -n` may differ on platforms.
The signatures are made against uncompressed .tar output, so this is not a
consideration.
Another fun one I saw recently: using export-subst with $Format:%h$ will
produce different results depending on how many objects are present in
the repository running git-archive.
As long as the output is the same with thhe flags we specified in the
comment, we're still okay. E.g.:
-----BEGIN PGP SIGNATURE-----
Comment: This signature is for the .tar version of the archive
Comment: git archive --format tar --prefix=linux-5.10.9/ v5.10.9
Comment: git version 2.30.0
If running "git archive --format tar --prefix=linux-5.10.9/ v5.10.9" becomes
non-deterministic within the same version of git, *then* I'm in trouble. It's
been remarkably stable within the past 8 years, so I don't expect there's a
dramatic reason why "--format tar" output would need to change -- it's not
like the tar spec is much of a moving target.
-K