This series attempts to fix "git archive" sometimes looking into
.gitattributes in working directory and may cause undesired effects.
Nguyá»
n Thái Ngá»c Duy (8):
archive: add shortcuts for --format and --prefix
This one is unrelated. Just a convenient patch.
move prune_cache() to git lib
archive: add a failure test wrt .gitattributes misreading
archive: add tests for directory selection
attr: add GIT_ATTR_INDEX "direction"
archive: use index instead of parsing tree directly
archive: disregard .gitattributes on working directory
Main part, which converts archive.c to load index first, then export from there.
archive: support creating archives from index
Well, show off patch ;-)
Documentation/git-archive.txt | 4 +-
archive.c | 106 +++++++++++++++++++++++++++++++----------
archive.h | 1 +
attr.c | 4 +-
attr.h | 3 +-
builtin-ls-files.c | 29 +-----------
cache.h | 2 +
read-cache.c | 27 ++++++++++
t/t0024-crlf-archive.sh | 35 +++++++++++++-
t/t5000-tar-tree.sh | 6 ++-
10 files changed, 158 insertions(+), 59 deletions(-)
"git archive" is used to create archives from a tree (or commit), so
it should not consult any .gitattributes files on working directory.
It currently does, so this patch records the failure.
Signed-off-by: Nguyễn Thái Ngọc Duy <redacted>
---
t/t0024-crlf-archive.sh | 7 +++++++
1 files changed, 7 insertions(+), 0 deletions(-)
@@ -43,4 +43,11 @@ test_expect_success UNZIP 'zip archive' ''+test_expect_failure'.gitattributes in workdir should not be consulted''+echo"sample -crlf">.gitattributes&&+gitarchive--format=tarHEAD|+(mkdiruntarred2&&cduntarred2&&"$TAR"-xf-)+test_cmpsampleuntarred2/sample+'+ test_done
While in archive.c, it says "pathspec". It's not really pathspec.
Make a few tests to catch this point.
Signed-off-by: Nguyễn Thái Ngọc Duy <redacted>
---
archive.c | 17 +++++++++++++++++
t/t0024-crlf-archive.sh | 30 +++++++++++++++++++++++++++++-
2 files changed, 46 insertions(+), 1 deletions(-)
@@ -9,14 +9,42 @@ test_expect_success setup 'gitconfigcore.autocrlftrue+mkdirsub&&printf"CRLF line ending\r\nAnd another\r\n">sample&&-gitaddsample&&+cpsamplesub/sample&&+gitaddsamplesub/sample&&test_tick&&gitcommit-mInitial'+cat<<\EOF>expected+sample+sub/+sub/sample+EOF++test_expect_success'archive without subdir''+gitarchive--format=dump-file-listHEAD>output&&+test_cmpoutputexpected+'++cat<<\EOF>expected+sub/+sub/sample+EOF++test_expect_success'archive with subdir''+gitarchive--format=dump-file-listHEADsub>output&&+test_cmpoutputexpected+'++test_expect_success'subdir is not pathspec''+test"$(gitarchive--format=dump-file-listHEADsub/)"=""+test"$(gitarchive--format=dump-file-listHEADs\*)"=""+'+ test_expect_success'tar archive''gitarchive--format=tarHEAD|
This instructs attr mechanism not to look into working .gitattributes
at all. Needed by tools that does not handle working directory, such
as "git archive".
Signed-off-by: Nguyễn Thái Ngọc Duy <redacted>
---
attr.c | 4 +++-
attr.h | 3 ++-
2 files changed, 5 insertions(+), 2 deletions(-)
@@ -9,7 +9,7 @@ git-archive - Create an archive of files from a named tree SYNOPSIS -------- [verse]-'git archive' --format=<fmt> [--list] [--prefix=<prefix>/] [<extra>]+'git archive' [-f|--format=<fmt>] [--list] [-p|--prefix=<prefix>/] [<extra>] [--output=<file>] [--remote=<repo> [--exec=<git-upload-archive>]] <tree-ish> [path...]
@@ -33,6 +33,7 @@ comment. OPTIONS -------+-f=<fmt>:: --format=<fmt>:: Format of the resulting archive: 'tar' or 'zip'. The default is 'tar'.
@@ -45,6 +46,7 @@ OPTIONS --verbose:: Report progress to stderr.+-p=<prefix>/:: --prefix=<prefix>/:: Prepend <prefix>/ to each filename in the archive.
@@ -260,8 +260,8 @@ static int parse_archive_args(int argc, const char **argv,intlist=0;structoptionopts[]={OPT_GROUP(""),-OPT_STRING(0,"format",&format,"fmt","archive format"),-OPT_STRING(0,"prefix",&base,"prefix",+OPT_STRING('f',"format",&format,"fmt","archive format"),+OPT_STRING('p',"prefix",&base,"prefix","prepend prefix to each pathname in the archive"),OPT_STRING(0,"output",&output,"file","write the archive to this file"),
@@ -455,6 +456,7 @@ extern int add_file_to_index(struct index_state *, const char *path, int flags);externstructcache_entry*make_cache_entry(unsignedintmode,constunsignedchar*sha1,constchar*path,intstage,intrefresh);externintce_same_name(structcache_entry*a,structcache_entry*b);externintindex_name_is_other(conststructindex_state*,constchar*,int);+externvoidprune_index(constchar*prefix,intprefix_len,structindex_state*istate);/* do stat comparison even if CE_VALID is true */#define CE_MATCH_IGNORE_VALID 01
Currently:
- "git archive" parses trees directly
- attr mechanism does not support in-tree attributes
Making attr.c support in-tree attributes is quite a non-trivial task,
IMHO. Instead this patch makes "git archive" read tree to index first,
then start exporting from there.
One minor regression: "git archive" now will no longer generate
directory entries, only files.
Signed-off-by: Nguyễn Thái Ngọc Duy <redacted>
---
archive.c | 66 ++++++++++++++++++++++++++++++++---------------
t/t0024-crlf-archive.sh | 2 -
2 files changed, 45 insertions(+), 23 deletions(-)
@@ -167,7 +169,9 @@ int write_archive_entries(struct archiver_args *args,write_archive_entry_fn_twrite_entry){structarchiver_contextcontext;-interr;+structunpack_trees_optionsopts;+structtree_desct;+inti,prefix_len,err;if(args->baselen>0&&args->base[args->baselen-1]=='/'){size_tlen=args->baselen;
@@ -185,11 +189,42 @@ int write_archive_entries(struct archiver_args *args,context.args=args;context.write_entry=write_entry;-err=read_tree_recursive(args->tree,args->base,args->baselen,0,-args->pathspec,write_archive_entry,&context);-if(err==READ_TREE_RECURSIVE)-err=0;-returnerr;+memset(&opts,0,sizeof(opts));+opts.index_only=1;+opts.head_idx=-1;+opts.src_index=&the_index;+opts.dst_index=&the_index;+opts.fn=oneway_merge;+init_tree_desc(&t,args->tree->buffer,args->tree->size);+if(unpack_trees(1,&t,&opts))+return-1;+prefix_len=common_prefix(args->pathspec);+if(prefix_len)+prune_cache(args->pathspec[0],prefix_len);++for(i=0;i<active_nr;i++){+structcache_entry*ce=active_cache[i];+constchar**path;+constchar*str,*prefix;+intmatch=0;++if(!args->pathspec||!args->pathspec[0]||!*args->pathspec[0])+match=1;++/* It's not really pathspec, so match_pathspec can't be used */+for(path=args->pathspec;!match&&*path;path++){+for(str=ce->name,prefix=*path;*prefix&&*str==*prefix;str++,prefix++)+;+if(!*prefix&&*str=='/')+match=1;+}++if(match)+write_archive_entry(ce->sha1,args->base,args->baselen,+ce->name,ce->ce_mode,ce_stage(ce),+&context);+}+return0;}staticconststructarchiver*lookup_archiver(constchar*name)
@@ -238,18 +274,6 @@ static void parse_treeish_arg(const char **argv,if(tree==NULL)die("not a tree object");-if(prefix){-unsignedchartree_sha1[20];-unsignedintmode;-interr;--err=get_tree_entry(tree->object.sha1,prefix,-tree_sha1,&mode);-if(err||!S_ISDIR(mode))-die("current working directory is untracked");--tree=parse_tree_indirect(tree_sha1);-}ar_args->tree=tree;ar_args->commit_sha1=commit_sha1;ar_args->commit=commit;
This is more or less for fun. Does anyone really want to create archives
from index?
Signed-off-by: Nguyễn Thái Ngọc Duy <redacted>
---
archive.c | 38 +++++++++++++++++++++++++-------------
archive.h | 1 +
2 files changed, 26 insertions(+), 13 deletions(-)
@@ -307,6 +312,8 @@ static int parse_archive_args(int argc, const char **argv,"prepend prefix to each pathname in the archive"),OPT_STRING(0,"output",&output,"file","write the archive to this file"),+OPT_BOOLEAN('i',"index",&use_index,+"generate from index instead"),OPT__VERBOSE(&verbose),OPT__COMPR('0',&compression_level,"store only",0),OPT__COMPR('1',&compression_level,"compress faster",1),
@@ -348,7 +355,7 @@ static int parse_archive_args(int argc, const char **argv,}/* We need at least one parameter -- tree-ish */-if(argc<1)+if(argc<1&&!use_index)usage_with_options(archive_usage,opts);*ar=lookup_archiver(format);if(!*ar)
Some tests in t500-tar-tree.sh depends on in-worktree
.gitattributes. Fix them too.
Signed-off-by: Nguyễn Thái Ngọc Duy <redacted>
---
We could make an option to keep old behaviour. But is it worth it?
archive.c | 1 +
t/t0024-crlf-archive.sh | 2 +-
t/t5000-tar-tree.sh | 6 +++++-
3 files changed, 7 insertions(+), 2 deletions(-)
@@ -69,7 +69,7 @@ test_expect_success UNZIP 'zip archive' ''-test_expect_failure'.gitattributes in workdir should not be consulted''+test_expect_success'.gitattributes in workdir should not be consulted''echo"sample -crlf">.gitattributes&&gitarchive--format=tarHEAD|(mkdiruntarred2&&cduntarred2&&"$TAR"-xf-)
From: Junio C Hamano <hidden> Date: 2016-06-15 22:46:35
Nguyễn Thái Ngọc Duy [off-list ref] writes:
"git archive" is used to create archives from a tree (or commit), so
it should not consult any .gitattributes files on working directory.
It currently does,...
...which _might_ be actively used as a _feature_ by people; I do not think
it is a bad idea to have a mode of operation where it solely works with
in-tree attributes ignoring the work tree ones, and I suspect we probably
would want to make that the default, but "so it should not" is probably a
bit too strong.
For example, an older version may record $Id$ with keyword expansion set
in its in-tree .gitattributes, and you later found it to be a mistake and
have removed it in the current version. Re-exporting an older version
with the current code wouldn't have $Id$ expanded, but with your patch it
will.
A stronger example would be crlf conversion, I suppose, but the same "with
the current code we can fix it up but after this patch we can't" worry
applies.
So perhaps you may want to resurrect the current behaviour with a new
option --fix-attributes, whose description would be "normally the command
takes attributes settings from the tree being exported, but .gitattributes
files in it may record wrong attributes you may want to fix while
exporting. With this option, corresponding .gitattributes files in the
work tree will override the in-tree .gitattributes" or something like
that.
From: Junio C Hamano <hidden> Date: 2016-06-15 22:46:35
Nguyễn Thái Ngọc Duy [off-list ref] writes:
Currently:
- "git archive" parses trees directly
- attr mechanism does not support in-tree attributes
Making attr.c support in-tree attributes is quite a non-trivial task,
IMHO. Instead this patch makes "git archive" read tree to index first,
then start exporting from there.
One minor regression: "git archive" now will no longer generate
directory entries, only files.
Hmmm, if you read_tree() into the_index upfront and do not change anything
else to the archive.c code, shouldn't it work without such a regression at
all? Am I missing something?
It would allow you to export the index into an archive, but I doubt it is
worth the amount of code churn.
From: Junio C Hamano <hidden> Date: 2016-06-15 22:46:35
Nguyễn Thái Ngọc Duy [off-list ref] writes:
Some tests in t500-tar-tree.sh depends on in-worktree
.gitattributes. Fix them too.
s/500/5000/;
You can also consider these as examples of possibly useful use cases of
the current "fix-attributes" feature you are breaking with this patch (see
my comments to 3/8).
From: René Scharfe <hidden> Date: 2016-06-15 22:46:35
Nguyễn Thái Ngọc Duy schrieb:
quoted hunk
+-f=<fmt>::
--format=<fmt>::
Format of the resulting archive: 'tar' or 'zip'. The default
is 'tar'.
@@ -45,6 +46,7 @@ OPTIONS --verbose:: Report progress to stderr.+-p=<prefix>/:: --prefix=<prefix>/:: Prepend <prefix>/ to each filename in the archive.
With short options, you can't use an equal sign to separate their
parameters (-f=<fmt> vs. -f<fmt>).
René
From: René Scharfe <hidden> Date: 2016-06-15 22:46:35
Nguyễn Thái Ngọc Duy schrieb:
quoted hunk
While in archive.c, it says "pathspec". It's not really pathspec.
Make a few tests to catch this point.
Signed-off-by: Nguyễn Thái Ngọc Duy <redacted>
---
archive.c | 17 +++++++++++++++++
t/t0024-crlf-archive.sh | 30 +++++++++++++++++++++++++++++-
2 files changed, 46 insertions(+), 1 deletions(-)
git archive dumps all its output, so I think "file-list" would be a
better name for an archiver creating lists of files.
However, if you only need it for tests (I can't imagine other uses at
the moment), you could do something like this instead:
$ git archive -v HEAD >/dev/null 2>file-list
It might be wasteful since it creates a full archive (including file
contents) and throws it away, but it's OK for testing purposes.
René
From: René Scharfe <hidden> Date: 2016-06-15 22:46:35
Nguyễn Thái Ngọc Duy schrieb:
Currently:
- "git archive" parses trees directly
- attr mechanism does not support in-tree attributes
Making attr.c support in-tree attributes is quite a non-trivial task,
IMHO. Instead this patch makes "git archive" read tree to index first,
then start exporting from there.
One minor regression: "git archive" now will no longer generate
directory entries, only files.
I don't like the need to prepare an index of all paths up front, but
that's just a gut feeling. I haven't looked into implementing in-tree
attribute support in attr.c; is it really that hard? Other commands
would benefit from this, too, right (e.g. any command using attributes
in a bare repo)?
René
From: René Scharfe <hidden> Date: 2016-06-15 22:46:36
Junio C Hamano schrieb:
Nguyễn Thái Ngọc Duy [off-list ref] writes:
quoted
"git archive" is used to create archives from a tree (or commit), so
it should not consult any .gitattributes files on working directory.
It currently does,...
...which _might_ be actively used as a _feature_ by people; I do not think
it is a bad idea to have a mode of operation where it solely works with
in-tree attributes ignoring the work tree ones, and I suspect we probably
would want to make that the default, but "so it should not" is probably a
bit too strong.
For example, an older version may record $Id$ with keyword expansion set
in its in-tree .gitattributes, and you later found it to be a mistake and
have removed it in the current version. Re-exporting an older version
with the current code wouldn't have $Id$ expanded, but with your patch it
will.
A stronger example would be crlf conversion, I suppose, but the same "with
the current code we can fix it up but after this patch we can't" worry
applies.
So perhaps you may want to resurrect the current behaviour with a new
option --fix-attributes, whose description would be "normally the command
takes attributes settings from the tree being exported, but .gitattributes
files in it may record wrong attributes you may want to fix while
exporting. With this option, corresponding .gitattributes files in the
work tree will override the in-tree .gitattributes" or something like
that.
Hmm, "fix" implies that something is broken without this option, which
is not necessarily the case. A purely descriptive name like
--worktree-attributes fits better IMHO.
Also, the last sentence of the description may be taken to imply a
mixing of attributes from work tree and exported tree is done, which is
not the case with the patch from series 2 (and shouldn't be). Perhaps:
"With this option, .gitattributes files are read from the work tree
instead of from the exported tree."
René
Also, the last sentence of the description may be taken to imply a
mixing of attributes from work tree and exported tree is done, which is
not the case with the patch from series 2 (and shouldn't be). Perhaps:
"With this option, .gitattributes files are read from the work tree
instead of from the exported tree."
I'm at work and haven't had chance to read other mails carefully. Wrt
to the description, it reads from index too if it fails to read from
worktree. So perhaps "With this option, .gitattributes files are read
from work tree (or from index if they are missing from work tree)
instead of from exported tree.", unless you want to introduce another
"attr direction" to read from work tree only.
--
Duy
From: René Scharfe <hidden> Date: 2016-06-15 22:46:36
Nguyen Thai Ngoc Duy schrieb:
2009/4/13 René Scharfe [off-list ref]:
quoted
Also, the last sentence of the description may be taken to imply a
mixing of attributes from work tree and exported tree is done, which is
not the case with the patch from series 2 (and shouldn't be). Perhaps:
"With this option, .gitattributes files are read from the work tree
instead of from the exported tree."
I'm at work and haven't had chance to read other mails carefully. Wrt
to the description, it reads from index too if it fails to read from
worktree. So perhaps "With this option, .gitattributes files are read
from work tree (or from index if they are missing from work tree)
instead of from exported tree.", unless you want to introduce another
"attr direction" to read from work tree only.
Err, yes, that was nonsense, please disregard this part. Sorry. I
somehow mixed up falling back to the index and merging the contents of
attribute files from both sources, not sure why. :-/
René