the git-archive manpage states:
"git archive [--format=<fmt>] [--list] [--prefix=<prefix>/] [<extra>] [-o
| --output=<file>] [--worktree-attributes] [--remote=<repo>
[--exec=<git-upload-archive>]] <tree-ish> [path\u2026]
<extra>
This can be any options that the archiver backend understands. See next
section."
I have tar 1.23 and want to use the --transform option. How can I feed
git-archive additional tar options?
Working syntax starting points for git-archive and tar:
git archive --format=tar -o my.tar HEAD Web/Templates/
tar -cvf my.tar --transform 's,^Web/Templates/,myPath/myWeb/Templates/,'
WebPortal/Templates/
Failed syntax attempts for feeding tar option to git-archive:
git archive --format=tar -o my.tar HEAD --transform
's,^Web/Templates/,myPath/myWeb/Templates/,' WebPortal/Templates/
error: unknown option `transform'
git archive --format=tar -o my.tar --transform
's,^Web/Templates/,myPath/myWeb/Templates/,' HEAD WebPortal/Templates/
error: unknown option `transform'
v/r,
neal
From: Jeff King <hidden> Date: 2016-06-15 22:51:35
On Wed, Jul 13, 2011 at 06:34:32PM -0500, Neal Kreitzinger wrote:
the git-archive manpage states:
"git archive [--format=<fmt>] [--list] [--prefix=<prefix>/] [<extra>] [-o
| --output=<file>] [--worktree-attributes] [--remote=<repo>
[--exec=<git-upload-archive>]] <tree-ish> [path\u2026]
<extra>
This can be any options that the archiver backend understands. See next
section."
I have tar 1.23 and want to use the --transform option. How can I feed
git-archive additional tar options?
Right. And the next section is "Backend Extra Options", which has:
zip
-0
Store the files instead of deflating them.
-9
Highest and slowest compression level. You can specify any number from 1 to 9
to adjust compression speed and ratio.
And nothing else. We don't actually call your system "tar" to generate
the tarball, which is what I assume you thought when you saw "backend".
A patch to make it more clear would be welcome.
Working syntax starting points for git-archive and tar:
git archive --format=tar -o my.tar HEAD Web/Templates/
tar -cvf my.tar --transform 's,^Web/Templates/,myPath/myWeb/Templates/,'
WebPortal/Templates/
Failed syntax attempts for feeding tar option to git-archive:
git archive --format=tar -o my.tar HEAD --transform
's,^Web/Templates/,myPath/myWeb/Templates/,' WebPortal/Templates/
error: unknown option `transform'
git archive --format=tar -o my.tar --transform
's,^Web/Templates/,myPath/myWeb/Templates/,' HEAD WebPortal/Templates/
error: unknown option `transform'
Yeah, that won't work, because there is no such option. We do have
"--prefix", but I suspect that's not flexible enough for what you want.
So you're probably stuck with extracting the results of "git archive" to
a temporary directory and then using GNU tar to re-archive them (or if
you have a checkout, you can just tar that up directly, feeding the list
from "git ls-files" into tar). It would be nice if GNU tar could act as
a post-processor, and do something like:
git archive HEAD | tar --pipe-mode --transform=whatever >my.tar
But AFAIK, nothing like "--pipe-mode" exists.
It would probably not be a very hard feature to add to "git archive" if
you're interested in doing so.
-Peff
From: René Scharfe <hidden> Date: 2016-06-15 22:51:35
Am 14.07.2011 03:56, schrieb Jeff King:
On Wed, Jul 13, 2011 at 06:34:32PM -0500, Neal Kreitzinger wrote:
quoted
Working syntax starting points for git-archive and tar:
git archive --format=tar -o my.tar HEAD Web/Templates/
tar -cvf my.tar --transform 's,^Web/Templates/,myPath/myWeb/Templates/,'
WebPortal/Templates/
Failed syntax attempts for feeding tar option to git-archive:
git archive --format=tar -o my.tar HEAD --transform
's,^Web/Templates/,myPath/myWeb/Templates/,' WebPortal/Templates/
error: unknown option `transform'
git archive --format=tar -o my.tar --transform
's,^Web/Templates/,myPath/myWeb/Templates/,' HEAD WebPortal/Templates/
error: unknown option `transform'
Yeah, that won't work, because there is no such option. We do have
"--prefix", but I suspect that's not flexible enough for what you want.
If you only need a single subdirectory with a custom prefix you could do
something like this (variables only used to keep the lines short):
$ subdir=WebPortal/Templates
$ prefix=myPath/myWeb/Templates/
$ (cd "$subdir" && git archive --prefix="$prefix" HEAD) >my.tar
The output file can be specified with -o as well, of course, but you'd
either need to use an absolute path or add "../" for each directory
level you descend into (-o ../../my.tar in this case).
René
Yeah, that won't work, because there is no such option. We do have
"--prefix", but I suspect that's not flexible enough for what you want.
If you only need a single subdirectory with a custom prefix you could do
something like this (variables only used to keep the lines short):
$ subdir=WebPortal/Templates
$ prefix=myPath/myWeb/Templates/
$ (cd "$subdir" && git archive --prefix="$prefix" HEAD) >my.tar
The output file can be specified with -o as well, of course, but you'd
either need to use an absolute path or add "../" for each directory
level you descend into (-o ../../my.tar in this case).
Couldn't you also do:
git archive --prefix=$prefix HEAD:$subdir >my.tar
? I guess that loses the pax header with the commit sha1 in it, though,
because you are feeding a straight tree instead of a commit.
We didn't when git-archive was written, but these days we have
get_sha1_with_context to remember incidental things about an object we
look up. It should perhaps remember the commit (if any) we used to reach
a treeish, and then the above command line could still insert the pax
header.
-Peff
Yeah, that won't work, because there is no such option. We do have
"--prefix", but I suspect that's not flexible enough for what you want.
If you only need a single subdirectory with a custom prefix you could do
something like this (variables only used to keep the lines short):
$ subdir=WebPortal/Templates
$ prefix=myPath/myWeb/Templates/
$ (cd "$subdir" && git archive --prefix="$prefix" HEAD) >my.tar
The output file can be specified with -o as well, of course, but you'd
either need to use an absolute path or add "../" for each directory
level you descend into (-o ../../my.tar in this case).
Couldn't you also do:
git archive --prefix=$prefix HEAD:$subdir >my.tar
? I guess that loses the pax header with the commit sha1 in it, though,
because you are feeding a straight tree instead of a commit.
Yes, and yes.
We didn't when git-archive was written, but these days we have
get_sha1_with_context to remember incidental things about an object we
look up. It should perhaps remember the commit (if any) we used to reach
a treeish, and then the above command line could still insert the pax
header.
That's a good idea to increase consistency, as there shouldn't really be
a difference in output between the two subdirectory syntaxes.
I always wondered, however, if the embedded commit ID has really been
used to identify the corresponding version of an archive that somehow
lost its filename (due to being piped?).
René
From: Jeff King <hidden> Date: 2016-06-15 22:51:35
On Thu, Jul 14, 2011 at 07:45:07PM +0200, René Scharfe wrote:
quoted
We didn't when git-archive was written, but these days we have
get_sha1_with_context to remember incidental things about an object we
look up. It should perhaps remember the commit (if any) we used to reach
a treeish, and then the above command line could still insert the pax
header.
That's a good idea to increase consistency, as there shouldn't really be
a difference in output between the two subdirectory syntaxes.
The patch to do this is pretty tiny. See below.
There are a few issues, though:
1. I think this is probably the right thing to do, and most people
will be happy about it. But I guess I can see an argument that the
commit-id should not be there, as the subtree does not represent
that commit.
IOW, if you assume the commit-id in the output means
"by the way, this came from commit X", this change is a good thing.
If you assume it means "this is the tree from commit X", then it's
not. I have no idea how people use it. I never have, but I always
assumed the use case was "I have this random tarball. Where did it
come from?".
2. The object_context already has the sha1 we want, but it is under
the name "tree", which is not an accurate name. It's actually
"whatever is on the left side of the :". Which should be a
tree-ish, but could be a commit or a tree.
3. It looks like we fill in object_context whenever we see something
like "tree-ish:path". But we should perhaps also do so when peeling
something like "tree-ish^{tree}".
I always wondered, however, if the embedded commit ID has really been
used to identify the corresponding version of an archive that somehow
lost its filename (due to being piped?).
I dunno. I've never used it.
-- >8 --
Subject: [PATCH] archive: look harder for commit id
When "git archive" is given a commit, the output will
contain the commit sha1 (either as a pax header for tar
format, or in a file comment for zip).
When it's given a name that resolves to a tree, like:
git archive git-1.7.0:Documentation
then the archive code never sees the commit, and no
commit-id is output. We can use get_sha1_with_context to
remember the commit that led us to that tree (if any).
Signed-off-by: Jeff King <redacted>
---
archive.c | 5 ++++-
1 files changed, 4 insertions(+), 1 deletions(-)
From: Jakub Narebski <hidden> Date: 2016-06-15 22:51:35
Jeff King [off-list ref] writes:
On Thu, Jul 14, 2011 at 07:45:07PM +0200, René Scharfe wrote:
quoted
quoted
We didn't when git-archive was written, but these days we have
get_sha1_with_context to remember incidental things about an object we
look up. It should perhaps remember the commit (if any) we used to reach
a treeish, and then the above command line could still insert the pax
header.
That's a good idea to increase consistency, as there shouldn't really be
a difference in output between the two subdirectory syntaxes.
The patch to do this is pretty tiny. See below.
There are a few issues, though:
1. I think this is probably the right thing to do, and most people
will be happy about it. But I guess I can see an argument that the
commit-id should not be there, as the subtree does not represent
that commit.
IOW, if you assume the commit-id in the output means
"by the way, this came from commit X", this change is a good thing.
If you assume it means "this is the tree from commit X", then it's
not. I have no idea how people use it. I never have, but I always
assumed the use case was "I have this random tarball. Where did it
come from?".
Perhaps we should embed '<commit-id>:<subtree>' instead in pax header,
in that case? Or <commit-id>.<subtree> if ':' is forbidden.
--
Jakub Narębski
Poland
Yeah, that won't work, because there is no such option. We do
have "--prefix", but I suspect that's not flexible enough for
what you want.
If you only need a single subdirectory with a custom prefix you
could do something like this (variables only used to keep the lines
short):
$ subdir=WebPortal/Templates $ prefix=myPath/myWeb/Templates/ $ (cd
"$subdir"&& git archive --prefix="$prefix" HEAD)>my.tar
The output file can be specified with -o as well, of course, but
you'd either need to use an absolute path or add "../" for each
directory level you descend into (-o ../../my.tar in this case).
Couldn't you also do:
git archive --prefix=$prefix HEAD:$subdir>my.tar
? I guess that loses the pax header with the commit sha1 in it,
though, because you are feeding a straight tree instead of a commit.
We didn't when git-archive was written, but these days we have
get_sha1_with_context to remember incidental things about an object
we look up. It should perhaps remember the commit (if any) we used to
reach a treeish, and then the above command line could still insert
the pax header.
HEAD:$subdir worked on my bare repo. I ran it for each transformant
pathspec and then combined the archives with tar --catenate:
# git archive --format=tar --prefix=myWeb/myRoot/myAPP/Templates/
HEAD:WebPortal/Templates/ >myAPP.myTag.tar
# git archive --format=tar --prefix=opt/mySTUFF/v01/SCRIPTS/
HEAD:SCRIPTS/ >SCRIPTS.tar
# tar --file=myAPP.myTag.tar -A SCRIPTS.tar
However, the permissions also need to change to 777 and tar --mode would
not effect this in combination with --catenation or -x. Is there a way
I can change the permissions without having to untar->chmod->retar, and
without having to use a non-bare repo as an intermediary?
v/r,
neal
From: René Scharfe <hidden> Date: 2016-06-15 22:51:37
Am 18.07.2011 20:13, schrieb Neal Kreitzinger:
HEAD:$subdir worked on my bare repo. I ran it for each transformant
pathspec and then combined the archives with tar --catenate:
# git archive --format=tar --prefix=myWeb/myRoot/myAPP/Templates/
HEAD:WebPortal/Templates/ >myAPP.myTag.tar
# git archive --format=tar --prefix=opt/mySTUFF/v01/SCRIPTS/
HEAD:SCRIPTS/ >SCRIPTS.tar
# tar --file=myAPP.myTag.tar -A SCRIPTS.tar
However, the permissions also need to change to 777 and tar --mode would
not effect this in combination with --catenation or -x. Is there a way
I can change the permissions without having to untar->chmod->retar, and
without having to use a non-bare repo as an intermediary?
You can use the configuration setting tar.umask to affect the
permissions of the archive entries. Set it to 0 to pass the permission
bits from the repo unchanged.
René
HEAD:$subdir worked on my bare repo. I ran it for each transformant
pathspec and then combined the archives with tar --catenate:
# git archive --format=tar --prefix=myWeb/myRoot/myAPP/Templates/
HEAD:WebPortal/Templates/>myAPP.myTag.tar
# git archive --format=tar --prefix=opt/mySTUFF/v01/SCRIPTS/
HEAD:SCRIPTS/>SCRIPTS.tar
# tar --file=myAPP.myTag.tar -A SCRIPTS.tar
However, the permissions also need to change to 777 and tar --mode would
not effect this in combination with --catenation or -x. Is there a way
I can change the permissions without having to untar->chmod->retar, and
without having to use a non-bare repo as an intermediary?
You can use the configuration setting tar.umask to affect the
permissions of the archive entries. Set it to 0 to pass the permission
bits from the repo unchanged.
The permissions in my repo are 775 and 664 and I want to change them to 777.
-neal
From: René Scharfe <hidden> Date: 2016-06-15 22:51:37
Am 19.07.2011 02:12, schrieb Neal Kreitzinger:
On 7/18/2011 3:50 PM, René Scharfe wrote:
quoted
Am 18.07.2011 20:13, schrieb Neal Kreitzinger:
quoted
However, the permissions also need to change to 777 and tar --mode would
not effect this in combination with --catenation or -x. Is there a way
I can change the permissions without having to untar->chmod->retar, and
without having to use a non-bare repo as an intermediary?
You can use the configuration setting tar.umask to affect the
permissions of the archive entries. Set it to 0 to pass the permission
bits from the repo unchanged.
The permissions in my repo are 775 and 664 and I want to change them to
777.
Git doesn't store all permission bits. If a file is marked as
executable then you get 777, otherwise 666 -- minus the umask, which is
0002 by default. So in order to achive rwx permissions for all in the
archive, you need to A) mark the files as executable in the repository
and B) set tar.umask to 0 to get allow the world to write.
However, what's the reason for requiring this lack of access control?
Why o+w?
René
However, the permissions also need to change to 777 and tar --mode would
not effect this in combination with --catenation or -x. Is there a way
I can change the permissions without having to untar->chmod->retar, and
without having to use a non-bare repo as an intermediary?
You can use the configuration setting tar.umask to affect the
permissions of the archive entries. Set it to 0 to pass the permission
bits from the repo unchanged.
The permissions in my repo are 775 and 664 and I want to change them to
777.
Git doesn't store all permission bits. If a file is marked as
executable then you get 777, otherwise 666 -- minus the umask, which is
0002 by default. So in order to achive rwx permissions for all in the
archive, you need to A) mark the files as executable in the repository
and B) set tar.umask to 0 to get allow the world to write.
However, what's the reason for requiring this lack of access control?
Why o+w?
tar.umask worked. Thank you for explaining how the permissions work in
this context. I now see that 775 and 664 would work for the apache
component and for executing our binaries. Thanks for pointing this
out. However, another element of our application is a proprietary
runtime that runs on top of linux and runs our core binaries. This
allows us to store our binaries in git and deploy them directly on the
customer server from git (via git-archive). That runtime needs o+w in
order to update the 'last run date' in the binary which is critical to
our troubleshooting in the field. o+w is needed because the user's
runtime instance runs with user permissions when executed from a linux
command line terminal and our users are not setup in the same group as
the binaries. Therefore, with tar.umask = 0000 I can deploy 777 and 666
permissions and everything will work.
I suppose I could write a script to change the tar.umask entry to 0000
only when running git-archive for the binary portion, and use tar.umask
0002 when extracting the other portions. I could also change our setup
to put the users and the runmodules in the same group and use tar.umask
0002 across the board. These would be more correct than the chmod 777
shotgun that we currently use to blast away our permissions problems.
git-archive is a "quick" solution to our immediate deployment needs.
Eventually, I plan on using git on the source and target machines as the
core mechanism to "promote to production" (ie. deploy to customer
servers). It looks like others are using git for deployment also. In
my previous shops which used other VCS's on minicomputers and
mainframes, "promote to production" meant the universal run path for all
users (and especially for productional data transactions) on that
central machine. In my current shop (my first linux shop) we have
multiple concurrent versions of production on a multitude of
productional machines and even concurrently on an individual
productional machine in some cases. The main reason we chose git is
because it is the only VCS that can handle this.
v/r,
neal
However, the permissions also need to change to 777 and tar
--mode would not effect this in combination with --catenation
or -x. Is there a way I can change the permissions without
having to untar->chmod->retar, and without having to use a
non-bare repo as an intermediary?
You can use the configuration setting tar.umask to affect the
permissions of the archive entries. Set it to 0 to pass the
permission bits from the repo unchanged.
The permissions in my repo are 775 and 664 and I want to change
them to 777.
Git doesn't store all permission bits. If a file is marked as
executable then you get 777, otherwise 666 -- minus the umask,
which is 0002 by default. So in order to achive rwx permissions for
all in the archive, you need to A) mark the files as executable in
the repository and B) set tar.umask to 0 to get allow the world to
write.
However, what's the reason for requiring this lack of access
control? Why o+w?
tar.umask worked. Thank you for explaining how the permissions work
in this context. I now see that 775 and 664 would work for the apache
component and for executing our binaries. Thanks for pointing this
out. However, another element of our application is a proprietary
runtime that runs on top of linux and runs our core binaries. This
allows us to store our binaries in git and deploy them directly on
the customer server from git (via git-archive). That runtime needs
o+w in order to update the 'last run date' in the binary which is
critical to our troubleshooting in the field. o+w is needed because
the user's runtime instance runs with user permissions when executed
from a linux command line terminal and our users are not setup in the
same group as the binaries. Therefore, with tar.umask = 0000 I can
deploy 777 and 666 permissions and everything will work.
I suppose I could write a script to change the tar.umask entry to
0000 only when running git-archive for the binary portion, and use
tar.umask 0002 when extracting the other portions. I could also
change our setup to put the users and the runmodules in the same
group and use tar.umask 0002 across the board. These would be more
correct than the chmod 777 shotgun that we currently use to blast
away our permissions problems.
git-archive is a "quick" solution to our immediate deployment needs.
Eventually, I plan on using git on the source and target machines as
the core mechanism to "promote to production" (ie. deploy to customer
servers). It looks like others are using git for deployment also. In
my previous shops which used other VCS's on minicomputers and
mainframes, "promote to production" meant the universal run path for
all users (and especially for productional data transactions) on that
central machine. In my current shop (my first linux shop) we have
multiple concurrent versions of production on a multitude of
productional machines and even concurrently on an individual
productional machine in some cases. The main reason we chose git is
because it is the only VCS that can handle this.
Actually, the apache user (web interface) also needs to be able to
update the binaries with 'last run date'. In this context the o+w allows
this, also. I don't know enough about apache and permissions to try and
add apache to the same group as the binaries at this point.
-neal