From: Brandon Williams <hidden> Date: 2017-01-13 00:00:55
This series has been bounced around a bit (from Junio to Stefan) and finally
landed in my lap. The end result of Stefan's attempt at the series still had a
couple of things that needed more tweaking. It also has a few patches on top
which added functionality to pathspecs to be able to query into the attribute
system, which I've dropped from this series due to this series' length.
As a reminder the intent of this series is to revamp the attribute system so
that it can be thread-safe as well as a couple of other quality of life
changes. This entailed removing dependencies on writing global data structures
during the attribute collection process. Major changes are as follows:
* The global array used to collect attributes needed to be made local and as a
result was pushed out to the attr_check structure the caller prepares before
querying the attribute system.
* As it turns out the attribute stack ends up being used as a read-only
structure during the collection process and as such parts of the attribute
stack can be shared between different threads calling into the system. To
enable this sharing the attribute stack frames are stored in a hashmap and
can be read out (or created and stored in the hashmap) based on the
directory name of the path being queried. This is possible because if a
particular stack frame is included in the overall stack for a particular
query, all of the frames underneath it will be the same for all queries that
use this frame (only exception is the info frame which is handled special
case, see the patch for details).
I took many of the first patches of this series as is from the series Stefan
prepared as as such may only need a cursory glace. I did modify and change
some of the later patches authored by Junio to address a couple of naming
changes and to redistribute some code between patches so those patches would
need a closer look.
Thanks again to all the work Junio and Stefan put into this before I got a hold
of it.
Any comments are appreciated!
Thanks,
Brandon Williams
Brandon Williams (8):
attr: pass struct attr_check to collect_some_attrs
attr: use hashmap for attribute dictionary
attr: eliminate global check_all_attr array
attr: remove maybe-real, maybe-macro from git_attr
attr: tighten const correctness with git_attr and match_attr
attr: store attribute stacks in hashmap
attr: push the bare repo check into read_attr()
attr: reformat git_attr_set_direction() function
Junio C Hamano (17):
commit.c: use strchrnul() to scan for one line
attr.c: use strchrnul() to scan for one line
attr.c: update a stale comment on "struct match_attr"
attr.c: explain the lack of attr-name syntax check in parse_attr()
attr.c: complete a sentence in a comment
attr.c: mark where #if DEBUG ends more clearly
attr.c: simplify macroexpand_one()
attr.c: tighten constness around "git_attr" structure
attr.c: plug small leak in parse_attr_line()
attr.c: add push_stack() helper
attr.c: outline the future plans by heavily commenting
attr: rename function and struct related to checking attributes
attr: (re)introduce git_check_attr() and struct attr_check
attr: convert git_all_attrs() to use "struct attr_check"
attr: convert git_check_attrs() callers to use the new API
attr: retire git_check_attrs() API
attr: change validity check for attribute names to use positive logic
Nguyễn Thái Ngọc Duy (1):
attr: support quoting pathname patterns in C style
Stefan Beller (1):
Documentation: fix a typo
Documentation/gitattributes.txt | 10 +-
Documentation/technical/api-gitattributes.txt | 86 ++-
archive.c | 24 +-
attr.c | 932 +++++++++++++++++---------
attr.h | 50 +-
builtin/check-attr.c | 66 +-
builtin/pack-objects.c | 19 +-
commit.c | 3 +-
common-main.c | 3 +
convert.c | 25 +-
ll-merge.c | 33 +-
t/t0003-attributes.sh | 26 +
userdiff.c | 19 +-
ws.c | 19 +-
14 files changed, 834 insertions(+), 481 deletions(-)
--
2.11.0.390.gc69c2f50cf-goog
From: Brandon Williams <hidden> Date: 2017-01-12 23:55:32
From: Junio C Hamano <redacted>
Convert 'invalid_attr_name()' to 'attr_name_valid()' and use positive
logic for the return value. In addition create a helper function that
prints out an error message when an invalid attribute name is used.
We could later update the message to exactly spell out what the
rules for a good attribute name are, etc.
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 34 ++++++++++++++++++++--------------
1 file changed, 20 insertions(+), 14 deletions(-)
@@ -74,23 +74,33 @@ static unsigned hash_name(const char *name, int namelen)returnval;}-staticintinvalid_attr_name(constchar*name,intnamelen)+staticintattr_name_valid(constchar*name,size_tnamelen){/**Attributenamecannotbeginwith'-'andmustconsistof*charactersfrom[-A-Za-z0-9_.].*/if(namelen<=0||*name=='-')-return-1;+return0;while(namelen--){charch=*name++;if(!(ch=='-'||ch=='.'||ch=='_'||('0'<=ch&&ch<='9')||('a'<=ch&&ch<='z')||('A'<=ch&&ch<='Z')))-return-1;+return0;}-return0;+return1;+}++staticvoidreport_invalid_attr(constchar*name,size_tlen,+constchar*src,intlineno)+{+structstrbuferr=STRBUF_INIT;+strbuf_addf(&err,_("%.*s is not a valid attribute name"),+(int)len,name);+fprintf(stderr,"%s: %s:%d\n",err.buf,src,lineno);+strbuf_release(&err);}staticstructgit_attr*git_attr_internal(constchar*name,intlen)
From: Brandon Williams <hidden> Date: 2017-01-12 23:55:35
The old callchain used to take an array of attr_check_item items.
Instead pass the 'attr_check' container object to 'collect_some_attrs()'
and access the fields in the data structure directly.
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 33 +++++++++++++--------------------
1 file changed, 13 insertions(+), 20 deletions(-)
@@ -777,9 +777,7 @@ static int macroexpand_one(int nr, int rem)*check_all_attr.Ifnumisnon-zero,onlyattributesincheck[]are*collected.Otherwiseallattributesarecollected.*/-staticvoidcollect_some_attrs(constchar*path,intnum,-structattr_check_item*check)-+staticvoidcollect_some_attrs(constchar*path,structattr_check*check){structattr_stack*stk;inti,pathlen,rem,dirlen;
From: Brandon Williams <hidden> Date: 2017-01-12 23:55:42
From: Junio C Hamano <redacted>
Since nobody uses the old API, make it file-scope static, and update
the documentation to describe the new API.
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
Documentation/technical/api-gitattributes.txt | 86 +++++++++++++++++----------
attr.c | 3 +-
attr.h | 1 -
3 files changed, 58 insertions(+), 32 deletions(-)
@@ -16,10 +16,15 @@ Data Structure of no interest to the calling programs. The name of the attribute can be retrieved by calling `git_attr_name()`.-`struct git_attr_check`::+`struct attr_check_item`::- This structure represents a set of attributes to check in a call- to `git_check_attr()` function, and receives the results.+ This structure represents one attribute and its value.++`struct attr_check`::++ This structure represents a collection of `attr_check_item`.+ It is passed to `git_check_attr()` function, specifying the+ attributes to check, and receives their values. Attribute Values
@@ -27,7 +32,7 @@ Attribute Values An attribute for a path can be in one of four states: Set, Unset, Unspecified or set to a string, and `.value` member of `struct-git_attr_check` records it. There are three macros to check these:+attr_check_item` records it. There are three macros to check these: `ATTR_TRUE()`::
@@ -48,49 +53,51 @@ value of the attribute for the path. Querying Specific Attributes -----------------------------* Prepare an array of `struct git_attr_check` to define the list of- attributes you would want to check. To populate this array, you would- need to define necessary attributes by calling `git_attr()` function.+* Prepare `struct attr_check` using attr_check_initl()+ function, enumerating the names of attributes whose values you are+ interested in, terminated with a NULL pointer. Alternatively, an+ empty `struct attr_check` can be prepared by calling+ `attr_check_alloc()` function and then attributes you want to+ ask about can be added to it with `attr_check_append()`+ function. * Call `git_check_attr()` to check the attributes for the path.-* Inspect `git_attr_check` structure to see how each of the attribute in- the array is defined for the path.+* Inspect `attr_check` structure to see how each of the+ attribute in the array is defined for the path. Example --------To see how attributes "crlf" and "indent" are set for different paths.+To see how attributes "crlf" and "ident" are set for different paths.-. Prepare an array of `struct git_attr_check` with two elements (because- we are checking two attributes). Initialize their `attr` member with- pointers to `struct git_attr` obtained by calling `git_attr()`:+. Prepare a `struct attr_check` with two elements (because+ we are checking two attributes): -------------static struct git_attr_check check[2];+static struct attr_check *check; static void setup_check(void) {- if (check[0].attr)+ if (check) return; /* already done */- check[0].attr = git_attr("crlf");- check[1].attr = git_attr("ident");+ check = attr_check_initl("crlf", "ident", NULL); } -------------. Call `git_check_attr()` with the prepared array of `struct git_attr_check`:+. Call `git_check_attr()` with the prepared `struct attr_check`: ------------ const char *path; setup_check();- git_check_attr(path, ARRAY_SIZE(check), check);+ git_check_attr(path, check); -------------. Act on `.value` member of the result, left in `check[]`:+. Act on `.value` member of the result, left in `check->check[]`: ------------- const char *value = check[0].value;+ const char *value = check->check[0].value; if (ATTR_TRUE(value)) { The attribute is Set, by listing only the name of the
@@ -109,20 +116,39 @@ static void setup_check(void) } ------------+To see how attributes in argv[] are set for different paths, only+the first step in the above would be different.++------------+static struct attr_check *check;+static void setup_check(const char **argv)+{+ check = attr_check_alloc();+ while (*argv) {+ struct git_attr *attr = git_attr(*argv);+ attr_check_append(check, attr);+ argv++;+ }+}+------------+ Querying All Attributes ----------------------- To get the values of all attributes associated with a file:-* Call `git_all_attrs()`, which returns an array of `git_attr_check`- structures.+* Prepare an empty `attr_check` structure by calling+ `attr_check_alloc()`.++* Call `git_all_attrs()`, which populates the `attr_check`+ with the attributes attached to the path.-* Iterate over the `git_attr_check` array to examine the attribute- names and values. The name of the attribute described by a- `git_attr_check` object can be retrieved via- `git_attr_name(check[i].attr)`. (Please note that no items will be- returned for unset attributes, so `ATTR_UNSET()` will return false- for all returned `git_array_check` objects.)+* Iterate over the `attr_check.check[]` array to examine+ the attribute names and values. The name of the attribute+ described by a `attr_check.check[]` object can be retrieved via+ `git_attr_name(check->check[i].attr)`. (Please note that no items+ will be returned for unset attributes, so `ATTR_UNSET()` will return+ false for all returned `attr_check.check[]` objects.)-* Free the `git_array_check` array.+* Free the `attr_check` struct by calling `attr_check_free()`.
From: Brandon Williams <hidden> Date: 2017-01-12 23:55:48
From: Junio C Hamano <redacted>
The remaining callers are all simple "I have N attributes I am
interested in. I'll ask about them with various paths one by one".
After this step, no caller to git_check_attrs() remains. After
removing it, we can extend "struct attr_check" struct with data
that can be used in optimizing the query for the specific N
attributes it contains.
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
archive.c | 24 ++++++------------------
builtin/pack-objects.c | 19 +++++--------------
convert.c | 17 ++++++-----------
ll-merge.c | 33 ++++++++++++++-------------------
userdiff.c | 19 ++++++++-----------
ws.c | 19 ++++++-------------
6 files changed, 45 insertions(+), 86 deletions(-)
From: Brandon Williams <hidden> Date: 2017-01-12 23:55:51
From: Junio C Hamano <redacted>
A common pattern to check N attributes for many paths is to
(1) prepare an array A of N attr_check_item items;
(2) call git_attr() to intern the N attribute names and fill A;
(3) repeatedly call git_check_attrs() for path with N and A;
A look-up for these N attributes for a single path P scans the
entire attr_stack, starting from the .git/info/attributes file and
then .gitattributes file in the directory the path P is in, going
upwards to find .gitattributes file found in parent directories.
An earlier commit 06a604e6 (attr: avoid heavy work when we know the
specified attr is not defined, 2014-12-28) tried to optimize out
this scanning for one trivial special case: when the attribute being
sought is known not to exist, we do not have to scan for it. While
this may be a cheap and effective heuristic, it would not work well
when N is (much) more than 1.
What we would want is a more customized way to skip irrelevant
entries in the attribute stack, and the definition of irrelevance
is tied to the set of attributes passed to git_check_attrs() call,
i.e. the set of attributes being sought. The data necessary for
this optimization needs to live alongside the set of attributes, but
a simple array of git_attr_check_elem simply does not have any place
for that.
Introduce "struct attr_check" that contains N, the number of
attributes being sought, and A, the array that holds N
attr_check_item items, and a function git_check_attr() that
takes a path P and this structure as its parameters. This structure
can later be extended to hold extra data necessary for optimization.
Also, to make it easier to write the first two steps in common
cases, introduce git_attr_check_initl() helper function, which takes
a NULL-terminated list of attribute names and initialize this
structure.
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
attr.h | 17 +++++++++++++++
2 files changed, 91 insertions(+)
@@ -865,6 +865,80 @@ int git_all_attrs(const char *path, int *num, struct attr_check_item **check)return0;}+structattr_check*attr_check_alloc(void)+{+returnxcalloc(1,sizeof(structattr_check));+}++intgit_check_attr(constchar*path,structattr_check*check)+{+returngit_check_attrs(path,check->check_nr,check->check);+}++structattr_check*attr_check_initl(constchar*one,...)+{+structattr_check*check;+intcnt;+va_listparams;+constchar*param;++va_start(params,one);+for(cnt=1;(param=va_arg(params,constchar*))!=NULL;cnt++)+;+va_end(params);++check=attr_check_alloc();+check->check_nr=cnt;+check->check_alloc=cnt;+check->check=xcalloc(cnt,sizeof(structattr_check_item));++check->check[0].attr=git_attr(one);+va_start(params,one);+for(cnt=1;cnt<check->check_nr;cnt++){+structgit_attr*attr;+param=va_arg(params,constchar*);+if(!param)+die("BUG: counted %d != ended at %d",+check->check_nr,cnt);+attr=git_attr(param);+if(!attr)+die("BUG: %s: not a valid attribute name",param);+check->check[cnt].attr=attr;+}+va_end(params);+returncheck;+}++structattr_check_item*attr_check_append(structattr_check*check,+conststructgit_attr*attr)+{+structattr_check_item*item;++ALLOC_GROW(check->check,check->check_nr+1,check->check_alloc);+item=&check->check[check->check_nr++];+item->attr=attr;+returnitem;+}++voidattr_check_reset(structattr_check*check)+{+check->check_nr=0;+}++voidattr_check_clear(structattr_check*check)+{+free(check->check);+check->check=NULL;+check->check_alloc=0;+check->check_nr=0;+}++voidattr_check_free(structattr_check*check)+{+attr_check_clear(check);+free(check);+}+voidgit_attr_set_direction(enumgit_attr_directionnew,structindex_state*istate){enumgit_attr_directionold=direction;
From: Brandon Williams <hidden> Date: 2017-01-12 23:55:53
From: Junio C Hamano <redacted>
This updates the other two ways the attribute check is done via an
array of "struct attr_check_item" elements. These two niches
appear only in "git check-attr".
* The caller does not know offhand what attributes it wants to ask
about and cannot use attr_check_initl() to prepare the
attr_check structure.
* The caller may not know what attributes it wants to ask at all,
and instead wants to learn everything that the given path has.
Such a caller can call attr_check_alloc() to allocate an empty
attr_check, and then call attr_check_append() to add attribute names
one by one.
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 38 ++++++++++++---------------------
attr.h | 9 +++-----
builtin/check-attr.c | 60 ++++++++++++++++++++++++++--------------------------
3 files changed, 47 insertions(+), 60 deletions(-)
@@ -837,42 +837,32 @@ int git_check_attrs(const char *path, int num, struct attr_check_item *check)return0;}-intgit_all_attrs(constchar*path,int*num,structattr_check_item**check)+voidgit_all_attrs(constchar*path,structattr_check*check){-inti,count,j;+inti;-collect_some_attrs(path,0,NULL);+attr_check_reset(check);+collect_some_attrs(path,check->check_nr,check->check);-/* Count the number of attributes that are set. */-count=0;-for(i=0;i<attr_nr;i++){-constchar*value=check_all_attr[i].value;-if(value!=ATTR__UNSET&&value!=ATTR__UNKNOWN)-++count;-}-*num=count;-ALLOC_ARRAY(*check,count);-j=0;for(i=0;i<attr_nr;i++){+constchar*name=check_all_attr[i].attr->name;constchar*value=check_all_attr[i].value;-if(value!=ATTR__UNSET&&value!=ATTR__UNKNOWN){-(*check)[j].attr=check_all_attr[i].attr;-(*check)[j].value=value;-++j;-}+structattr_check_item*item;+if(value==ATTR__UNSET||value==ATTR__UNKNOWN)+continue;+item=attr_check_append(check,git_attr(name));+item->value=value;}--return0;}-structattr_check*attr_check_alloc(void)+intgit_check_attr(constchar*path,structattr_check*check){-returnxcalloc(1,sizeof(structattr_check));+returngit_check_attrs(path,check->check_nr,check->check);}-intgit_check_attr(constchar*path,structattr_check*check)+structattr_check*attr_check_alloc(void){-returngit_check_attrs(path,check->check_nr,check->check);+returnxcalloc(1,sizeof(structattr_check));}structattr_check*attr_check_initl(constchar*one,...)
From: Brandon Williams <hidden> Date: 2017-01-12 23:55:54
Whether or not a git attribute is real or a macro isn't a property of
the attribute but rather it depends on the attribute stack (which
.gitattribute files were read).
This patch removes the 'maybe_real' and 'maybe_macro' fields in a
git_attr and instead adds the 'macro' field to a attr_check_item. The
'macro' indicates (if non-NULL) that a particular attribute is a macro
for the given attribute stack. It's populated, through a quick scan of
the attribute stack, with the match_attr that corresponds to the macro's
definition. This way the attribute stack only needs to be scanned a
single time prior to attribute collection instead of each time a macro
needs to be expanded.
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 69 ++++++++++++++++++++++++++++++------------------------------------
attr.h | 6 ++++++
2 files changed, 37 insertions(+), 38 deletions(-)
@@ -418,10 +405,6 @@ static struct match_attr *parse_attr_line(const char *line, const char *src,/* Second pass to fill the attr_states */for(cp=states,i=0;*cp;i++){cp=parse_attr(src,lineno,cp,&(res->state[i]));-if(!is_macro)-res->state[i].attr->maybe_real=1;-if(res->state[i].attr->maybe_macro)-cannot_trust_maybe_real=1;}strbuf_release(&pattern);
@@ -826,7 +809,7 @@ static int path_matches(const char *pathname, int pathlen,staticintmacroexpand_one(structattr_check_item*all_attrs,intnr,intrem);staticintfill_one(constchar*what,structattr_check_item*all_attrs,-structmatch_attr*a,intrem)+conststructmatch_attr*a,intrem){inti;
@@ -867,24 +850,34 @@ static int fill(const char *path, int pathlen, int basename_offset,staticintmacroexpand_one(structattr_check_item*all_attrs,intnr,intrem){-structattr_stack*stk;-inti;+conststructattr_check_item*item=&all_attrs[nr];-if(all_attrs[nr].value!=ATTR__TRUE||-!all_attrs[nr].attr->maybe_macro)+if(item->macro&&item->value==ATTR__TRUE)+returnfill_one("expand",all_attrs,item->macro,rem);+elsereturnrem;+}-for(stk=attr_stack;stk;stk=stk->prev){-for(i=stk->num_matches-1;0<=i;i--){-structmatch_attr*ma=stk->attrs[i];-if(!ma->is_macro)-continue;-if(ma->u.attr->attr_nr==nr)-returnfill_one("expand",all_attrs,ma,rem);+/*+*Markstheattributeswhicharemacrosbasedontheattributestack.+*Thispreventshavingtosearchthroughtheattributestackeachtime+*amacroneedstobeexpandedduringthefillstage.+*/+staticvoiddetermine_macros(structattr_check_item*all_attrs,+conststructattr_stack*stack)+{+for(;stack;stack=stack->prev){+inti;+for(i=stack->num_matches-1;i>=0;i--){+conststructmatch_attr*ma=stack->attrs[i];+if(ma->is_macro){+intn=ma->u.attr->attr_nr;+if(!all_attrs[n].macro){+all_attrs[n].macro=ma;+}+}}}--returnrem;}/*
@@ -233,14 +233,14 @@ static struct git_attr *git_attr_internal(const char *name, int namelen)returna;}-structgit_attr*git_attr(constchar*name)+conststructgit_attr*git_attr(constchar*name){returngit_attr_internal(name,strlen(name));}/* What does a matched pattern decide? */structattr_state{-structgit_attr*attr;+conststructgit_attr*attr;constchar*setto;};
@@ -838,7 +838,7 @@ static int fill(const char *path, int pathlen, int basename_offset,constchar*base=stk->origin?stk->origin:"";for(i=stk->num_matches-1;0<rem&&0<=i;i--){-structmatch_attr*a=stk->attrs[i];+conststructmatch_attr*a=stk->attrs[i];if(a->is_macro)continue;if(path_matches(path,pathlen,basename_offset,
From: Brandon Williams <hidden> Date: 2017-01-12 23:55:59
The current implementation of the attribute dictionary uses a custom
hashtable. This modernizes the dictionary by converting it to the builtin
'hashmap' structure.
Also, in order to enable a threaded API in the future add an
accompanying mutex which must be acquired prior to accessing the
dictionary of interned attributes.
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 171 ++++++++++++++++++++++++++++++++++++++++++----------------
attr.h | 2 +
common-main.c | 3 ++
3 files changed, 131 insertions(+), 45 deletions(-)
@@ -23,28 +24,17 @@ static const char git_attr__unknown[] = "(builtin)unknown";#define ATTR__UNSET NULL#define ATTR__UNKNOWN git_attr__unknown-/* This is a randomly chosen prime. */-#define HASHSIZE 257-#ifndef DEBUG_ATTR#define DEBUG_ATTR 0#endif-/*-*NEEDSWORK:theglobaldictionaryoftheinternedattributes-*muststayasingletonevenafterwebecomethread-ready.-*Accesstothesemustbesurroundedwithmutexwhenithappens.-*/structgit_attr{-structgit_attr*next;-unsignedh;-intattr_nr;+intattr_nr;/* unique attribute number */intmaybe_macro;intmaybe_real;-charname[FLEX_ARRAY];+charname[FLEX_ARRAY];/* attribute name */};staticintattr_nr;-staticstructgit_attr*(git_attr_hash[HASHSIZE]);/**NEEDSWORK:maybe-real,maybe-macroarenotpropertyof
@@ -63,15 +53,94 @@ const char *git_attr_name(const struct git_attr *attr)returnattr->name;}-staticunsignedhash_name(constchar*name,intnamelen)+structattr_hashmap{+structhashmapmap;+#ifndef NO_PTHREADS+pthread_mutex_tmutex;+#endif+};++staticinlinevoidhashmap_lock(structattr_hashmap*map){-unsignedval=0,c;+#ifndef NO_PTHREADS+pthread_mutex_lock(&map->mutex);+#endif+}-while(namelen--){-c=*name++;-val=((val<<7)|(val>>22))^c;-}-returnval;+staticinlinevoidhashmap_unlock(structattr_hashmap*map)+{+#ifndef NO_PTHREADS+pthread_mutex_unlock(&map->mutex);+#endif+}++/*+*Theglobaldictionaryofallinternedattributes.This+*isasingletonobjectwhichissharedbetweenthreads.+*Accesstothisdictionarymustbesurroundedwithamutex.+*/+staticstructattr_hashmapg_attr_hashmap;++/* The container for objects stored in "struct attr_hashmap" */+structattr_hash_entry{+structhashmap_entryent;/* must be the first member! */+constchar*key;/* the key; memory should be owned by value */+size_tkeylen;/* length of the key */+void*value;/* the stored value */+};++/* attr_hashmap comparison function */+staticintattr_hash_entry_cmp(conststructattr_hash_entry*a,+conststructattr_hash_entry*b,+void*unused)+{+return(a->keylen!=b->keylen)||strncmp(a->key,b->key,a->keylen);+}++/* Initialize an 'attr_hashmap' object */+voidattr_hashmap_init(structattr_hashmap*map)+{+hashmap_init(&map->map,(hashmap_cmp_fn)attr_hash_entry_cmp,0);+}++/*+*Retrievethe'value'storedinahashmapgiventheprovided'key'.+*Ifthereisnomatchingentry,returnNULL.+*/+staticvoid*attr_hashmap_get(structattr_hashmap*map,+constchar*key,size_tkeylen)+{+structattr_hash_entryk;+structattr_hash_entry*e;++if(!map->map.tablesize)+attr_hashmap_init(map);++hashmap_entry_init(&k,memhash(key,keylen));+k.key=key;+k.keylen=keylen;+e=hashmap_get(&map->map,&k,NULL);++returne?e->value:NULL;+}++/* Add 'value' to a hashmap based on the provided 'key'. */+staticvoidattr_hashmap_add(structattr_hashmap*map,+constchar*key,size_tkeylen,+void*value)+{+structattr_hash_entry*e;++if(!map->map.tablesize)+attr_hashmap_init(map);++e=xmalloc(sizeof(structattr_hash_entry));+hashmap_entry_init(e,memhash(key,keylen));+e->key=key;+e->keylen=keylen;+e->value=value;++hashmap_add(&map->map,e);}staticintattr_name_valid(constchar*name,size_tnamelen)
From: Brandon Williams <hidden> Date: 2017-01-12 23:56:00
The last big hurdle towards a thread-safe API for the attribute system
is the reliance on a global attribute stack that is modified during each
call into the attribute system.
This patch removes this global stack and instead a stack is retrieved or
constructed locally. Since each of these stacks is only used as a
read-only structure once constructed, they can be stored in a hashmap
and shared between threads. The key into the hashmap of attribute
stacks is, in the general case, the directory that corresponds to the
attribute stack frame. For the core stack frames (builtin, system,
home, and info) a key of ".git/<name>-attr" is used to prevent potential
collisions since a directory or file named ".git" is disallowed.
One caveat with storing and sharing the stack frames like this is that
the info stack needs to be treated separately from the rest of the
attribute stack. This is because each stack frame holds a pointer to
the stack that comes before it and if it was placed on top of the rest
of the attribute stack then this pointer would be different for each
attribute stack and wouldn't be able to be shared between threads. In
order to allow for sharing the info stack frame it needs to be its own
isolated frame and can simply be processed first to have the same affect
of being at the top of the stack.
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 375 +++++++++++++++++++++++++++++++++++++++++------------------------
1 file changed, 235 insertions(+), 140 deletions(-)
@@ -434,17 +434,19 @@ static struct match_attr *parse_attr_line(const char *line, const char *src,*.gitignorefileandinfo/excludesfileasafallback.*/-/* NEEDSWORK: This will become per git_attr_check */-staticstructattr_stack{-structattr_stack*prev;+structattr_stack{+conststructattr_stack*prev;char*origin;size_toriginlen;unsignednum_matches;unsignedalloc;structmatch_attr**attrs;-}*attr_stack;+};++/* Dictionary of stack frames; access should be surrounded by mutex */+staticstructattr_hashmapg_stack_hashmap;-staticvoidfree_attr_elem(structattr_stack*e)+staticvoidattr_stack_free(structattr_stack*e){inti;free(e->origin);
@@ -645,142 +665,208 @@ static int git_attr_system(void)staticGIT_PATH_FUNC(git_path_info_attributes,INFOATTRIBUTES_FILE)-staticvoidpush_stack(structattr_stack**attr_stack_p,-structattr_stack*elem,char*origin,size_toriginlen)+/*+*Thisfuncitonshouldonlybecalledfrom'get_attr_stack()'or+*'get_info_stack()',whichalreadyneedstoacquirethelocktothestack+*hashmap,sothereisnoneedtoalsoacquirethelockinthisfunction.+*/+staticvoidpush_stack(conststructattr_stack**attr_stack_p,+structattr_stack*elem,+constchar*origin,size_toriginlen){if(elem){-elem->origin=origin;-if(origin)-elem->originlen=originlen;+elem->origin=xmemdupz(origin,originlen);+elem->originlen=originlen;elem->prev=*attr_stack_p;*attr_stack_p=elem;+attr_hashmap_add(&g_stack_hashmap,elem->origin,+elem->originlen,elem);}}-staticvoidbootstrap_attr_stack(void)+/*+*Returnthepathbasethatcanbeusedinthepatternmatchingoperation.In+*ordertoenablestoringthecoreandinfostackframesinthestackhashmap+*anoriginstringotherthanNULLneededtobeused.Sincegitdisallows+*trackinga".git"fileordirectorythecoreandinfostackframeshavean+*originstringof".git/<frame>"andmustbeconvertedtotheemptystring+*whenbeingusedtopatternmatch.+*/+staticconstchar*attr_stack_get_base(conststructattr_stack*stack,+size_t*baselen){-structattr_stack*elem;+constchar*base;-if(attr_stack)-return;+if(starts_with(stack->origin,".git/")){+base="";+*baselen=0;+}else{+base=stack->origin;+*baselen=stack->originlen;+}-push_stack(&attr_stack,read_attr_from_array(builtin_attr),NULL,0);+returnbase;+}-if(git_attr_system())-push_stack(&attr_stack,-read_attr_from_file(git_etc_gitattributes(),1),-NULL,0);+/*+*Atthebottomoftheattributestackisthebuilt-in+*setofattributedefinitions,followedbythecontents+*of$(prefix)/etc/gitattributesandafilespecifiedby+*core.attributesfile.Then,contentsfrom+*.gitattributefilesfromdirectoriesclosertothe+*roottotheonesindeeperdirectoriesarepushed+*tothestack.Finally,attheverytopofthestack+*wealwayskeepthecontentsof$GIT_DIR/info/attributes.+*+*Whenchecking,weuseentriesfromnearthetopofthe+*stack,preferring$GIT_DIR/info/attributes,then+*.gitattributesindeeperdirectoriestoshallowerones,+*andfinallyusethebuilt-insetasthedefault.+*+*Theinfostackneedstobetreatedseparatelyfromtherestoftheattribute+*stack.Thisisbecauseeachstackframeholdsapointertothestackthat+*comesbeforeitandifitwasplacedontopoftherestoftheattribute+*stackthenthispointerwouldbedifferentforeachattributestackand+*wouldn'tbeabletobesharedbetweenthreads.Iftheinfostackistobe+*sharedthenitneedstobeitsownisolatedframeandcansimplybe+*processedfirsttohavethesameaffectofbeingatthetopofthestack.+*/+staticconststructattr_stack*get_info_stack(void)+{+conststructattr_stack*info;+constchar*key=".git/info-attr";+size_tkeylen=strlen(key);-if(!git_attributes_file)-git_attributes_file=xdg_config_home("attributes");-if(git_attributes_file)-push_stack(&attr_stack,-read_attr_from_file(git_attributes_file,1),-NULL,0);--if(!is_bare_repository()||direction==GIT_ATTR_INDEX){-elem=read_attr(GITATTRIBUTES_FILE,1);-push_stack(&attr_stack,elem,xstrdup(""),0);-debug_push(elem);+hashmap_lock(&g_stack_hashmap);++info=attr_hashmap_get(&g_stack_hashmap,key,keylen);++if(!info){+structattr_stack*e=NULL;++if(startup_info->have_repository)+e=read_attr_from_file(git_path_info_attributes(),1);++if(!e)+e=xcalloc(1,sizeof(structattr_stack));+e->origin=xstrdup(key);+e->originlen=keylen;++attr_hashmap_add(&g_stack_hashmap,e->origin,e->originlen,e);+info=e;}-if(startup_info->have_repository)-elem=read_attr_from_file(git_path_info_attributes(),1);-else-elem=NULL;+hashmap_unlock(&g_stack_hashmap);-if(!elem)-elem=xcalloc(1,sizeof(*elem));-push_stack(&attr_stack,elem,NULL,0);+returninfo;}-staticvoidprepare_attr_stack(constchar*path,intdirlen)+/*+*Thisfuncitonshouldonlybecalledfrom'get_attr_stack()',whichalready+*needstoacquirethelocktothestackhashmap,sothereisnoneedtoalso+*acquirethelockinthisfunction.+*/+staticconststructattr_stack*core_attr_stack(void){-structattr_stack*elem,*info;-constchar*cp;+conststructattr_stack*core;-/*-*Atthebottomoftheattributestackisthebuilt-in-*setofattributedefinitions,followedbythecontents-*of$(prefix)/etc/gitattributesandafilespecifiedby-*core.attributesfile.Then,contentsfrom-*.gitattributefilesfromdirectoriesclosertothe-*roottotheonesindeeperdirectoriesarepushed-*tothestack.Finally,attheverytopofthestack-*wealwayskeepthecontentsof$GIT_DIR/info/attributes.-*-*Whenchecking,weuseentriesfromnearthetopofthe-*stack,preferring$GIT_DIR/info/attributes,then-*.gitattributesindeeperdirectoriestoshallowerones,-*andfinallyusethebuilt-insetasthedefault.-*/-bootstrap_attr_stack();+core=attr_hashmap_get(&g_stack_hashmap,"",0);-/*-*Popthe"info"onethatisalwaysatthetopofthestack.-*/-info=attr_stack;-attr_stack=info->prev;+if(!core){+structattr_stack*e;+constchar*key;-/*-*Poptheonesfromdirectoriesthatarenottheprefixof-*thepathwearechecking.Breakoutoftheloopwhenwesee-*therootone(whoseoriginisanemptystring"")orthebuiltin-*one(whoseoriginisNULL)withoutpoppingit.-*/-while(attr_stack->origin){-intnamelen=strlen(attr_stack->origin);--elem=attr_stack;-if(namelen<=dirlen&&-!strncmp(elem->origin,path,namelen)&&-(!namelen||path[namelen]=='/'))-break;--debug_pop(elem);-attr_stack=elem->prev;-free_attr_elem(elem);-}+/* builtin frame */+e=read_attr_from_array(builtin_attr);+key=".git/builtin-attr";+push_stack(&core,e,key,strlen(key));-/*-*Readfromparentdirectoriesandpushthemdown-*/-if(!is_bare_repository()||direction==GIT_ATTR_INDEX){-/*-*bootstrap_attr_stack()shouldhaveadded,andthe-*aboveloopshouldhavestoppedbeforepopping,the-*rootelementwhoseattr_stack->originissettoan-*emptystring.-*/-structstrbufpathbuf=STRBUF_INIT;--assert(attr_stack->origin);-while(1){-size_tlen=strlen(attr_stack->origin);-char*origin;--if(dirlen<=len)-break;-cp=memchr(path+len+1,'/',dirlen-len-1);-if(!cp)-cp=path+dirlen;-strbuf_addf(&pathbuf,-"%.*s/%s",(int)(cp-path),path,-GITATTRIBUTES_FILE);-elem=read_attr(pathbuf.buf,0);-strbuf_setlen(&pathbuf,cp-path);-origin=strbuf_detach(&pathbuf,&len);-push_stack(&attr_stack,elem,origin,len);-debug_push(elem);+/* system-wide frame */+if(git_attr_system()){+e=read_attr_from_file(git_etc_gitattributes(),1);+key=".git/system-attr";+push_stack(&core,e,key,strlen(key));}-strbuf_release(&pathbuf);+/* home directory */+if(get_home_gitattributes()){+e=read_attr_from_file(get_home_gitattributes(),1);+key=".git/home-attr";+push_stack(&core,e,key,strlen(key));+}++/* root directory */+if(!is_bare_repository()||direction==GIT_ATTR_INDEX){+e=read_attr(GITATTRIBUTES_FILE,1);+}else{+e=xcalloc(1,sizeof(structattr_stack));+}+key="";+push_stack(&core,e,key,strlen(key));}-/*-*Finallypushthe"info"oneatthetopofthestack.-*/-push_stack(&attr_stack,info,NULL,0);+assert(core);+returncore;+}++staticconststructattr_stack*get_attr_stack(constchar*path,intdirlen)+{+conststructattr_stack*stack=NULL;+structstrbufkey=STRBUF_INIT;++strbuf_addstr(&key,path);++hashmap_lock(&g_stack_hashmap);++/* Search for the deepest, pre-constructed stack frame */+while(key.len&&!stack){+size_tlen=key.len;++/* Find start of the last component */+while(len>0&&!is_dir_sep(key.buf[len-1]))+len--;+/* Skip path-separator */+if(len>0&&is_dir_sep(key.buf[len-1]))+len--;+strbuf_setlen(&key,len);++stack=attr_hashmap_get(&g_stack_hashmap,key.buf,key.len);+}++/* At least start with the core stack */+if(!stack){+stack=core_attr_stack();+}++/* Build up to the directory 'path' is in */+while(key.len<dirlen){+size_tlen=key.len;+structattr_stack*next;++/* Skip path-separator */+if(len<dirlen&&is_dir_sep(path[len]))+len++;+/* Find the end of the next component */+while(len<dirlen&&!is_dir_sep(path[len]))+len++;++if(key.len>0)+strbuf_addch(&key,'/');+strbuf_add(&key,path+key.len,(len-key.len));+strbuf_addf(&key,"/%s",GITATTRIBUTES_FILE);++next=read_attr(key.buf,0);++/* reset the keybuffer to not include "/.gitattributes" */+strbuf_setlen(&key,len);++push_stack(&stack,next,key.buf,key.len);+}++hashmap_unlock(&g_stack_hashmap);++strbuf_release(&key);+returnstack;}staticintpath_matches(constchar*pathname,intpathlen,
From: Stefan Beller <hidden> Date: 2017-01-18 20:49:05
On Thu, Jan 12, 2017 at 3:53 PM, Brandon Williams [off-list ref] wrote:
-static void prepare_attr_stack(const char *path, int dirlen)
+/*
+ * This funciton should only be called from 'get_attr_stack()', which already
"function"
+ /* system-wide frame */
+ if (git_attr_system()) {
+ e = read_attr_from_file(git_etc_gitattributes(), 1);
read_attr_from_file may return NULL, so we'd have to treat this similar
to below "root directory", i.e. xcalloc for an empty frame?
+
+ /* root directory */
+ if (!is_bare_repository() || direction == GIT_ATTR_INDEX) {
+ e = read_attr(GITATTRIBUTES_FILE, 1);
+ } else {
+ e = xcalloc(1, sizeof(struct attr_stack));
+ }
+ key = "";
+ push_stack(&core, e, key, strlen(key));
If this is a bare repo, could we just omit this frame instead of pushing
an empty xcalloc'd frame? (Same for the stack frames of system wide
and home dir) ?
From: Brandon Williams <hidden> Date: 2017-01-18 20:50:33
On 01/18, Stefan Beller wrote:
On Thu, Jan 12, 2017 at 3:53 PM, Brandon Williams [off-list ref] wrote:
quoted
-static void prepare_attr_stack(const char *path, int dirlen)
+/*
+ * This funciton should only be called from 'get_attr_stack()', which already
"function"
quoted
+ /* system-wide frame */
+ if (git_attr_system()) {
+ e = read_attr_from_file(git_etc_gitattributes(), 1);
read_attr_from_file may return NULL, so we'd have to treat this similar
to below "root directory", i.e. xcalloc for an empty frame?
The push_stack function doesn't do anything if 'e' is NULL, so we should
be fine here.
quoted
+
+ /* root directory */
+ if (!is_bare_repository() || direction == GIT_ATTR_INDEX) {
+ e = read_attr(GITATTRIBUTES_FILE, 1);
+ } else {
+ e = xcalloc(1, sizeof(struct attr_stack));
+ }
+ key = "";
+ push_stack(&core, e, key, strlen(key));
If this is a bare repo, could we just omit this frame instead of pushing
an empty xcalloc'd frame? (Same for the stack frames of system wide
and home dir) ?
The reasoning behind having the object created even if its a bare repo
is so that later we can easily see that a frame has been read and
included and doesn't need to attempt to reread the frame from disk
later. It also made things simpler when storing the object in a hashmap
since storing a NULL ptr was awkward.
Though looking at Junio's discussion we may want to rethink how the
stacks are handled. I still need to think about it some more.
--
Brandon Williams
From: Stefan Beller <hidden> Date: 2017-01-18 21:21:48
On Wed, Jan 18, 2017 at 12:39 PM, Stefan Beller [off-list ref] wrote:
On Thu, Jan 12, 2017 at 3:53 PM, Brandon Williams [off-list ref] wrote:
quoted
-static void prepare_attr_stack(const char *path, int dirlen)
+/*
+ * This funciton should only be called from 'get_attr_stack()', which already
"function"
quoted
+ /* system-wide frame */
+ if (git_attr_system()) {
+ e = read_attr_from_file(git_etc_gitattributes(), 1);
read_attr_from_file may return NULL, so we'd have to treat this similar
to below "root directory", i.e. xcalloc for an empty frame?
quoted
+
+ /* root directory */
+ if (!is_bare_repository() || direction == GIT_ATTR_INDEX) {
+ e = read_attr(GITATTRIBUTES_FILE, 1);
+ } else {
+ e = xcalloc(1, sizeof(struct attr_stack));
+ }
+ key = "";
+ push_stack(&core, e, key, strlen(key));
If this is a bare repo, could we just omit this frame instead of pushing
an empty xcalloc'd frame? (Same for the stack frames of system wide
and home dir) ?
The next patch moves this issue into the read_attr function.
So in the end we'd either need to fix read_attr_from_file to return
res = xcalloc(1, sizeof(*res));
if (!fp), or we need to handle NULLs appropriately in 'core_attr_stack' ?
From: Brandon Williams <hidden> Date: 2017-01-12 23:56:04
Currently there is a reliance on 'check_all_attr' which is a global
array of 'attr_check_item' items which is used to store the value of
each attribute during the collection process.
This patch eliminates this global and instead creates an array per
'attr_check' instance which is then used in the attribute collection
process. This brings the attribute system one step closer to being
thread-safe.
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 114 +++++++++++++++++++++++++++++++++++++++++++----------------------
attr.h | 2 ++
2 files changed, 78 insertions(+), 38 deletions(-)
@@ -34,7 +34,6 @@ struct git_attr {intmaybe_real;charname[FLEX_ARRAY];/* attribute name */};-staticintattr_nr;/**NEEDSWORK:maybe-real,maybe-macroarenotpropertyof
@@ -45,9 +44,6 @@ static int attr_nr;*/staticintcannot_trust_maybe_real;-/* NEEDSWORK: This will become per git_attr_check */-staticstructattr_check_item*check_all_attr;-constchar*git_attr_name(conststructgit_attr*attr){returnattr->name;
@@ -791,16 +823,16 @@ static int path_matches(const char *pathname, int pathlen,pattern,prefix,pat->patternlen,pat->flags);}-staticintmacroexpand_one(intattr_nr,intrem);+staticintmacroexpand_one(structattr_check_item*all_attrs,intnr,intrem);-staticintfill_one(constchar*what,structmatch_attr*a,intrem)+staticintfill_one(constchar*what,structattr_check_item*all_attrs,+structmatch_attr*a,intrem){-structattr_check_item*check=check_all_attr;inti;-for(i=a->num_attr-1;0<rem&&0<=i;i--){+for(i=a->num_attr-1;rem>0&&i>=0;i--){structgit_attr*attr=a->state[i].attr;-constchar**n=&(check[attr->attr_nr].value);+constchar**n=&(all_attrs[attr->attr_nr].value);constchar*v=a->state[i].setto;if(*n==ATTR__UNKNOWN){
@@ -809,14 +841,15 @@ static int fill_one(const char *what, struct match_attr *a, int rem)attr,v);*n=v;rem--;-rem=macroexpand_one(attr->attr_nr,rem);+rem=macroexpand_one(all_attrs,attr->attr_nr,rem);}}returnrem;}staticintfill(constchar*path,intpathlen,intbasename_offset,-structattr_stack*stk,intrem)+structattr_stack*stk,structattr_check_item*all_attrs,+intrem){inti;constchar*base=stk->origin?stk->origin:"";
@@ -827,18 +860,18 @@ static int fill(const char *path, int pathlen, int basename_offset,continue;if(path_matches(path,pathlen,basename_offset,&a->u.pat,base,stk->originlen))-rem=fill_one("fill",a,rem);+rem=fill_one("fill",all_attrs,a,rem);}returnrem;}-staticintmacroexpand_one(intnr,intrem)+staticintmacroexpand_one(structattr_check_item*all_attrs,intnr,intrem){structattr_stack*stk;inti;-if(check_all_attr[nr].value!=ATTR__TRUE||-!check_all_attr[nr].attr->maybe_macro)+if(all_attrs[nr].value!=ATTR__TRUE||+!all_attrs[nr].attr->maybe_macro)returnrem;for(stk=attr_stack;stk;stk=stk->prev){
@@ -847,7 +880,7 @@ static int macroexpand_one(int nr, int rem)if(!ma->is_macro)continue;if(ma->u.attr->attr_nr==nr)-returnfill_one("expand",ma,rem);+returnfill_one("expand",all_attrs,ma,rem);}}
@@ -855,9 +888,9 @@ static int macroexpand_one(int nr, int rem)}/*-*Collectattributesforpathintothearraypointedtoby-*check_all_attr.Ifnumisnon-zero,onlyattributesincheck[]are-*collected.Otherwiseallattributesarecollected.+*Collectattributesforpathintothearraypointedtobycheck->all_attrs.+*Ifcheck->check_nrisnon-zero,onlyattributesincheck[]arecollected.+*Otherwiseallattributesarecollected.*/staticvoidcollect_some_attrs(constchar*path,structattr_check*check){
From: Brandon Williams <hidden> Date: 2017-01-12 23:56:05
Move the 'git_attr_set_direction()' up to be closer to the variables
that it modifies as well as a small formatting by renaming the variable
'new' to 'new_direction' so that it is more descriptive.
Update the comment about how 'direction' is used to read the state of
the world. It should be noted that callers of
'git_attr_set_direction()' should ensure that other threads are not
making calls into the attribute system until after the call to
'git_attr_set_direction()' completes. This function essentially acts as
reset button for the attribute system and should be handled with care.
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 49 ++++++++++++++++++++-----------------------------
attr.h | 3 ++-
2 files changed, 22 insertions(+), 30 deletions(-)
@@ -521,26 +521,30 @@ static struct attr_stack *read_attr_from_array(const char **list)}/*-*NEEDSWORK:thesetwoaretricky.Thecallersassumethereisa-*single,system-wideglobalstate"where we read attributes from?"-*andwhenthestateisflippedbycallinggit_attr_set_direction(),-*attr_stackisdiscardedsothatsubsequentattr_checkwilllazily-*readfromtherightplace.Andtheydonotknoworcarewhocalled-*bythemusestheattributesubsystem,hencehavenoknowledgeof-*existinggit_attr_checkinstancesorfutureonesthatwillbe-*created).-*-*Probablyweneedathread_localthatholdsthesetwovariables,-*andalistofgit_attr_checkinstances(whichneedtobemaintained-*byhookingintogit_attr_check_alloc(),git_attr_check_initl(),and-*git_attr_check_clear().Thengit_attr_set_direction()updatesthe-*fieldsinthatthread_localforthesetwovariables,iterateover-*alltheactivegit_attr_checkinstancesanddiscardtheattr_stack-*theyhold.Yuck,butitsoundsdoable.+*Callersintotheattributesystemassumethereisasingle,system-wide+*globalstatewhereattributesarereadfromandwhenthestateisflippedby+*callinggit_attr_set_direction(),thestackframesthathavebeen+*constructedneedtobediscardedsosothatsubsequentcallsintothe+*attributesystemwilllazilyreadfromtherightplace.Sincechanging+*directioncausesaglobalparadigmshift,itshouldnoteverbecalledwhile+*anotherthreadcouldpotentiallybecallingintotheattributesystem.*/staticenumgit_attr_directiondirection;staticstructindex_state*use_index;+voidgit_attr_set_direction(enumgit_attr_directionnew_direction,+structindex_state*istate)+{+if(is_bare_repository()&&new_direction!=GIT_ATTR_INDEX)+die("BUG: non-INDEX attr direction in a bare repo");++if(new_direction!=direction)+drop_attr_stack();++direction=new_direction;+use_index=istate;+}+staticstructattr_stack*read_attr_from_file(constchar*path,intmacro_ok){FILE*fp=fopen(path,"r");
@@ -1130,19 +1134,6 @@ void attr_check_free(struct attr_check *check)free(check);}-voidgit_attr_set_direction(enumgit_attr_directionnew,structindex_state*istate)-{-enumgit_attr_directionold=direction;--if(is_bare_repository()&&new!=GIT_ATTR_INDEX)-die("BUG: non-INDEX attr direction in a bare repo");--direction=new;-if(new!=old)-drop_attr_stack();-use_index=istate;-}-voidattr_start(void){pthread_mutex_init(&g_attr_hashmap.mutex,NULL);
From: Brandon Williams <hidden> Date: 2017-01-12 23:56:06
Push the bare repository check into the 'read_attr()' function. This
avoids needing to have extra logic which creates an empty stack frame
when inside a bare repo as a similar bit of logic already exists in the
'read_attr()' function.
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 40 ++++++++++++++++++++--------------------
1 file changed, 20 insertions(+), 20 deletions(-)
From: Brandon Williams <hidden> Date: 2017-01-13 00:00:29
From: Junio C Hamano <redacted>
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 40 +++++++++++++++++++++++++++++++++++++++-
1 file changed, 39 insertions(+), 1 deletion(-)
@@ -39,10 +44,19 @@ struct git_attr {charname[FLEX_ARRAY];};staticintattr_nr;+staticstructgit_attr*(git_attr_hash[HASHSIZE]);++/*+*NEEDSWORK:maybe-real,maybe-macroarenotpropertyof+*anattribute,asitdependsonwhat.gitattributesare+*read.Onceweintroducepergit_attr_checkattr_stack+*andcheck_all_attr,theoptimizationbasedonthemwill+*becomeunnecessaryandcangoaway.Soisthisvariable.+*/staticintcannot_trust_maybe_real;+/* NEEDSWORK: This will become per git_attr_check */staticstructgit_attr_check*check_all_attr;-staticstructgit_attr*(git_attr_hash[HASHSIZE]);constchar*git_attr_name(conststructgit_attr*attr){
@@ -318,6 +337,7 @@ static struct match_attr *parse_attr_line(const char *line, const char *src,*.gitignorefileandinfo/excludesfileasafallback.*/+/* NEEDSWORK: This will become per git_attr_check */staticstructattr_stack{structattr_stack*prev;char*origin;
@@ -88,7 +88,7 @@ is either not set or empty, $HOME/.config/git/attributes is used instead. Attributes for all users on a system should be placed in the `$(prefix)/etc/gitattributes` file.-Sometimes you would need to override an setting of an attribute+Sometimes you would need to override a setting of an attribute for a path to `Unspecified` state. This can be done by listing the name of the attribute prefixed with an exclamation point `!`.
From: Brandon Williams <hidden> Date: 2017-01-13 00:00:43
From: Junio C Hamano <redacted>
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
commit.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
From: Brandon Williams <hidden> Date: 2017-01-13 00:00:45
From: Junio C Hamano <redacted>
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
From: Brandon Williams <hidden> Date: 2017-01-13 00:00:48
From: Junio C Hamano <redacted>
There are too many repetitious "I have this new attr_stack element;
push it at the top of the stack" sequence. The new helper function
push_stack() gives us a way to express what is going on at these
places, and as a side effect, halves the number of times we mention
the attr_stack global variable.
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 71 +++++++++++++++++++++++++++++++-----------------------------------
1 file changed, 33 insertions(+), 38 deletions(-)
From: Brandon Williams <hidden> Date: 2017-01-13 00:00:51
From: Junio C Hamano <redacted>
It holds an interned string, and git_attr_name() is a way to peek
into it. Make sure the involved pointer types are pointer-to-const.
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 2 +-
attr.h | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)
From: Brandon Williams <hidden> Date: 2017-01-13 00:01:43
From: Junio C Hamano <redacted>
The traditional API to check attributes is to prepare an N-element
array of "struct git_attr_check" and pass N and the array to the
function "git_check_attr()" as arguments.
In preparation to revamp the API to pass a single structure, in
which these N elements are held, rename the type used for these
individual array elements to "struct attr_check_item" and rename
the function to "git_check_attrs()".
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
archive.c | 6 +++---
attr.c | 12 ++++++------
attr.h | 8 ++++----
builtin/check-attr.c | 19 ++++++++++---------
builtin/pack-objects.c | 6 +++---
convert.c | 12 ++++++------
ll-merge.c | 10 +++++-----
userdiff.c | 4 ++--
ws.c | 6 +++---
9 files changed, 42 insertions(+), 41 deletions(-)
@@ -56,7 +56,7 @@ static struct git_attr *(git_attr_hash[HASHSIZE]);staticintcannot_trust_maybe_real;/* NEEDSWORK: This will become per git_attr_check */-staticstructgit_attr_check*check_all_attr;+staticstructattr_check_item*check_all_attr;constchar*git_attr_name(conststructgit_attr*attr){
@@ -713,7 +713,7 @@ static int macroexpand_one(int attr_nr, int rem);staticintfill_one(constchar*what,structmatch_attr*a,intrem){-structgit_attr_check*check=check_all_attr;+structattr_check_item*check=check_all_attr;inti;for(i=a->num_attr-1;0<rem&&0<=i;i--){
@@ -778,7 +778,7 @@ static int macroexpand_one(int nr, int rem)*collected.Otherwiseallattributesarecollected.*/staticvoidcollect_some_attrs(constchar*path,intnum,-structgit_attr_check*check)+structattr_check_item*check){structattr_stack*stk;
From: Brandon Williams <hidden> Date: 2017-01-13 00:01:46
From: Junio C Hamano <redacted>
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
From: Brandon Williams <hidden> Date: 2017-01-13 00:02:05
From: Junio C Hamano <redacted>
The double-loop wants to do an early return immediately when one
matching macro is found. Eliminate the extra variable 'a' used for
that purpose and rewrite the "assign the found item to 'a' to make
it non-NULL and force the loop(s) to terminate" with a direct return
from there.
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 11 ++++-------
1 file changed, 4 insertions(+), 7 deletions(-)
@@ -705,24 +705,21 @@ static int fill(const char *path, int pathlen, int basename_offset,staticintmacroexpand_one(intnr,intrem){structattr_stack*stk;-structmatch_attr*a=NULL;inti;if(check_all_attr[nr].value!=ATTR__TRUE||!check_all_attr[nr].attr->maybe_macro)returnrem;-for(stk=attr_stack;!a&&stk;stk=stk->prev)-for(i=stk->num_matches-1;!a&&0<=i;i--){+for(stk=attr_stack;stk;stk=stk->prev){+for(i=stk->num_matches-1;0<=i;i--){structmatch_attr*ma=stk->attrs[i];if(!ma->is_macro)continue;if(ma->u.attr->attr_nr==nr)-a=ma;+returnfill_one("expand",ma,rem);}--if(a)-rem=fill_one("expand",a,rem);+}returnrem;}
From: Brandon Williams <hidden> Date: 2017-01-13 00:02:22
From: Nguyễn Thái Ngọc Duy <redacted>
Full pattern must be quoted. So 'pat"t"ern attr' will give exactly
'pat"t"ern', not 'pattern'. Also clarify that leading whitespaces are
not part of the pattern and document comment syntax.
Signed-off-by: Nguyễn Thái Ngọc Duy <redacted>
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
Documentation/gitattributes.txt | 8 +++++---
attr.c | 15 +++++++++++++--
t/t0003-attributes.sh | 26 ++++++++++++++++++++++++++
3 files changed, 44 insertions(+), 5 deletions(-)
@@ -21,9 +21,11 @@ Each line in `gitattributes` file is of form: pattern attr1 attr2 ... That is, a pattern followed by an attributes list,-separated by whitespaces. When the pattern matches the-path in question, the attributes listed on the line are given to-the path.+separated by whitespaces. Leading and trailing whitespaces are+ignored. Lines that begin with '#' are ignored. Patterns+that begin with a double quote are quoted in C style.+When the pattern matches the path in question, the attributes+listed on the line are given to the path. Each attribute can be in one of these states for a given path:
From: Brandon Williams <hidden> Date: 2017-01-13 00:02:26
From: Junio C Hamano <redacted>
If any error is noticed after the match_attr structure is allocated,
we shouldn't just return NULL from this function.
Add a fail_return label that frees the allocated structure and
returns NULL, and consistently jump there when we want to return
NULL after cleaning up.
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
From: Brandon Williams <hidden> Date: 2017-01-13 00:02:31
From: Junio C Hamano <redacted>
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 6 ++++++
1 file changed, 6 insertions(+)
From: Brandon Williams <hidden> Date: 2017-01-13 00:02:37
From: Junio C Hamano <redacted>
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
From: Brandon Williams <hidden> Date: 2017-01-13 00:03:28
From: Junio C Hamano <redacted>
When 82dce998 (attr: more matching optimizations from .gitignore,
2012-10-15) changed a pointer to a string "*pattern" into an
embedded "struct pattern" in struct match_attr, it forgot to update
the comment that describes the structure.
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
From: Brandon Williams <hidden> Date: 2017-01-23 20:35:43
Changes in v2:
* surround the mutex initializer calls by #ifdef
* mark file-local symbol static
* handling of attribute stacks. Instead of storing each stack frame in a
hashmap, there is a stack per attr_check instance. This will allow for
easier optimizing of the stack in future patches as well as eliminates the
potential for memory to grow unbounded. This is also more inline with the
original vision of the attribute system refactor.
Brandon Williams (8):
attr: pass struct attr_check to collect_some_attrs
attr: use hashmap for attribute dictionary
attr: eliminate global check_all_attr array
attr: remove maybe-real, maybe-macro from git_attr
attr: tighten const correctness with git_attr and match_attr
attr: store attribute stack in attr_check structure
attr: push the bare repo check into read_attr()
attr: reformat git_attr_set_direction() function
Junio C Hamano (17):
commit.c: use strchrnul() to scan for one line
attr.c: use strchrnul() to scan for one line
attr.c: update a stale comment on "struct match_attr"
attr.c: explain the lack of attr-name syntax check in parse_attr()
attr.c: complete a sentence in a comment
attr.c: mark where #if DEBUG ends more clearly
attr.c: simplify macroexpand_one()
attr.c: tighten constness around "git_attr" structure
attr.c: plug small leak in parse_attr_line()
attr.c: add push_stack() helper
attr.c: outline the future plans by heavily commenting
attr: rename function and struct related to checking attributes
attr: (re)introduce git_check_attr() and struct attr_check
attr: convert git_all_attrs() to use "struct attr_check"
attr: convert git_check_attrs() callers to use the new API
attr: retire git_check_attrs() API
attr: change validity check for attribute names to use positive logic
Nguyễn Thái Ngọc Duy (1):
attr: support quoting pathname patterns in C style
Stefan Beller (1):
Documentation: fix a typo
Documentation/gitattributes.txt | 10 +-
Documentation/technical/api-gitattributes.txt | 86 ++-
archive.c | 24 +-
attr.c | 854 ++++++++++++++++++--------
attr.h | 53 +-
builtin/check-attr.c | 66 +-
builtin/pack-objects.c | 19 +-
commit.c | 3 +-
common-main.c | 3 +
convert.c | 25 +-
ll-merge.c | 33 +-
t/t0003-attributes.sh | 26 +
userdiff.c | 19 +-
ws.c | 19 +-
14 files changed, 800 insertions(+), 440 deletions(-)
--
2.11.0.483.g087da7b7c-goog
From: Brandon Williams <hidden> Date: 2017-01-23 20:35:46
From: Junio C Hamano <redacted>
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
commit.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
From: Brandon Williams <hidden> Date: 2017-01-23 20:35:48
From: Junio C Hamano <redacted>
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
From: Brandon Williams <hidden> Date: 2017-01-23 20:35:51
From: Junio C Hamano <redacted>
When 82dce998 (attr: more matching optimizations from .gitignore,
2012-10-15) changed a pointer to a string "*pattern" into an
embedded "struct pattern" in struct match_attr, it forgot to update
the comment that describes the structure.
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
From: Brandon Williams <hidden> Date: 2017-01-23 20:35:52
From: Junio C Hamano <redacted>
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 6 ++++++
1 file changed, 6 insertions(+)
From: Brandon Williams <hidden> Date: 2017-01-23 20:35:55
From: Junio C Hamano <redacted>
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
From: Brandon Williams <hidden> Date: 2017-01-23 20:35:56
From: Junio C Hamano <redacted>
The double-loop wants to do an early return immediately when one
matching macro is found. Eliminate the extra variable 'a' used for
that purpose and rewrite the "assign the found item to 'a' to make
it non-NULL and force the loop(s) to terminate" with a direct return
from there.
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 11 ++++-------
1 file changed, 4 insertions(+), 7 deletions(-)
@@ -705,24 +705,21 @@ static int fill(const char *path, int pathlen, int basename_offset,staticintmacroexpand_one(intnr,intrem){structattr_stack*stk;-structmatch_attr*a=NULL;inti;if(check_all_attr[nr].value!=ATTR__TRUE||!check_all_attr[nr].attr->maybe_macro)returnrem;-for(stk=attr_stack;!a&&stk;stk=stk->prev)-for(i=stk->num_matches-1;!a&&0<=i;i--){+for(stk=attr_stack;stk;stk=stk->prev){+for(i=stk->num_matches-1;0<=i;i--){structmatch_attr*ma=stk->attrs[i];if(!ma->is_macro)continue;if(ma->u.attr->attr_nr==nr)-a=ma;+returnfill_one("expand",ma,rem);}--if(a)-rem=fill_one("expand",a,rem);+}returnrem;}
From: Brandon Williams <hidden> Date: 2017-01-23 20:36:01
From: Nguyễn Thái Ngọc Duy <redacted>
Full pattern must be quoted. So 'pat"t"ern attr' will give exactly
'pat"t"ern', not 'pattern'. Also clarify that leading whitespaces are
not part of the pattern and document comment syntax.
Signed-off-by: Nguyễn Thái Ngọc Duy <redacted>
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
Documentation/gitattributes.txt | 8 +++++---
attr.c | 15 +++++++++++++--
t/t0003-attributes.sh | 26 ++++++++++++++++++++++++++
3 files changed, 44 insertions(+), 5 deletions(-)
@@ -21,9 +21,11 @@ Each line in `gitattributes` file is of form: pattern attr1 attr2 ... That is, a pattern followed by an attributes list,-separated by whitespaces. When the pattern matches the-path in question, the attributes listed on the line are given to-the path.+separated by whitespaces. Leading and trailing whitespaces are+ignored. Lines that begin with '#' are ignored. Patterns+that begin with a double quote are quoted in C style.+When the pattern matches the path in question, the attributes+listed on the line are given to the path. Each attribute can be in one of these states for a given path:
@@ -88,7 +88,7 @@ is either not set or empty, $HOME/.config/git/attributes is used instead. Attributes for all users on a system should be placed in the `$(prefix)/etc/gitattributes` file.-Sometimes you would need to override an setting of an attribute+Sometimes you would need to override a setting of an attribute for a path to `Unspecified` state. This can be done by listing the name of the attribute prefixed with an exclamation point `!`.
From: Brandon Williams <hidden> Date: 2017-01-23 20:36:05
From: Junio C Hamano <redacted>
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 40 +++++++++++++++++++++++++++++++++++++++-
1 file changed, 39 insertions(+), 1 deletion(-)
@@ -39,10 +44,19 @@ struct git_attr {charname[FLEX_ARRAY];};staticintattr_nr;+staticstructgit_attr*(git_attr_hash[HASHSIZE]);++/*+*NEEDSWORK:maybe-real,maybe-macroarenotpropertyof+*anattribute,asitdependsonwhat.gitattributesare+*read.Onceweintroducepergit_attr_checkattr_stack+*andcheck_all_attr,theoptimizationbasedonthemwill+*becomeunnecessaryandcangoaway.Soisthisvariable.+*/staticintcannot_trust_maybe_real;+/* NEEDSWORK: This will become per git_attr_check */staticstructgit_attr_check*check_all_attr;-staticstructgit_attr*(git_attr_hash[HASHSIZE]);constchar*git_attr_name(conststructgit_attr*attr){
@@ -318,6 +337,7 @@ static struct match_attr *parse_attr_line(const char *line, const char *src,*.gitignorefileandinfo/excludesfileasafallback.*/+/* NEEDSWORK: This will become per git_attr_check */staticstructattr_stack{structattr_stack*prev;char*origin;
From: Brandon Williams <hidden> Date: 2017-01-23 20:36:09
From: Junio C Hamano <redacted>
If any error is noticed after the match_attr structure is allocated,
we shouldn't just return NULL from this function.
Add a fail_return label that frees the allocated structure and
returns NULL, and consistently jump there when we want to return
NULL after cleaning up.
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)
From: Brandon Williams <hidden> Date: 2017-01-23 20:36:13
From: Junio C Hamano <redacted>
There are too many repetitious "I have this new attr_stack element;
push it at the top of the stack" sequence. The new helper function
push_stack() gives us a way to express what is going on at these
places, and as a side effect, halves the number of times we mention
the attr_stack global variable.
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 71 +++++++++++++++++++++++++++++++-----------------------------------
1 file changed, 33 insertions(+), 38 deletions(-)
From: Brandon Williams <hidden> Date: 2017-01-23 20:36:16
From: Junio C Hamano <redacted>
The traditional API to check attributes is to prepare an N-element
array of "struct git_attr_check" and pass N and the array to the
function "git_check_attr()" as arguments.
In preparation to revamp the API to pass a single structure, in
which these N elements are held, rename the type used for these
individual array elements to "struct attr_check_item" and rename
the function to "git_check_attrs()".
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
archive.c | 6 +++---
attr.c | 12 ++++++------
attr.h | 8 ++++----
builtin/check-attr.c | 19 ++++++++++---------
builtin/pack-objects.c | 6 +++---
convert.c | 12 ++++++------
ll-merge.c | 10 +++++-----
userdiff.c | 4 ++--
ws.c | 6 +++---
9 files changed, 42 insertions(+), 41 deletions(-)
@@ -56,7 +56,7 @@ static struct git_attr *(git_attr_hash[HASHSIZE]);staticintcannot_trust_maybe_real;/* NEEDSWORK: This will become per git_attr_check */-staticstructgit_attr_check*check_all_attr;+staticstructattr_check_item*check_all_attr;constchar*git_attr_name(conststructgit_attr*attr){
@@ -713,7 +713,7 @@ static int macroexpand_one(int attr_nr, int rem);staticintfill_one(constchar*what,structmatch_attr*a,intrem){-structgit_attr_check*check=check_all_attr;+structattr_check_item*check=check_all_attr;inti;for(i=a->num_attr-1;0<rem&&0<=i;i--){
@@ -778,7 +778,7 @@ static int macroexpand_one(int nr, int rem)*collected.Otherwiseallattributesarecollected.*/staticvoidcollect_some_attrs(constchar*path,intnum,-structgit_attr_check*check)+structattr_check_item*check){structattr_stack*stk;
From: Brandon Williams <hidden> Date: 2017-01-23 20:36:18
From: Junio C Hamano <redacted>
Convert 'invalid_attr_name()' to 'attr_name_valid()' and use positive
logic for the return value. In addition create a helper function that
prints out an error message when an invalid attribute name is used.
We could later update the message to exactly spell out what the
rules for a good attribute name are, etc.
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 34 ++++++++++++++++++++--------------
1 file changed, 20 insertions(+), 14 deletions(-)
@@ -74,23 +74,33 @@ static unsigned hash_name(const char *name, int namelen)returnval;}-staticintinvalid_attr_name(constchar*name,intnamelen)+staticintattr_name_valid(constchar*name,size_tnamelen){/**Attributenamecannotbeginwith'-'andmustconsistof*charactersfrom[-A-Za-z0-9_.].*/if(namelen<=0||*name=='-')-return-1;+return0;while(namelen--){charch=*name++;if(!(ch=='-'||ch=='.'||ch=='_'||('0'<=ch&&ch<='9')||('a'<=ch&&ch<='z')||('A'<=ch&&ch<='Z')))-return-1;+return0;}-return0;+return1;+}++staticvoidreport_invalid_attr(constchar*name,size_tlen,+constchar*src,intlineno)+{+structstrbuferr=STRBUF_INIT;+strbuf_addf(&err,_("%.*s is not a valid attribute name"),+(int)len,name);+fprintf(stderr,"%s: %s:%d\n",err.buf,src,lineno);+strbuf_release(&err);}staticstructgit_attr*git_attr_internal(constchar*name,intlen)
From: Brandon Williams <hidden> Date: 2017-01-23 20:36:21
The current implementation of the attribute dictionary uses a custom
hashtable. This modernizes the dictionary by converting it to the builtin
'hashmap' structure.
Also, in order to enable a threaded API in the future add an
accompanying mutex which must be acquired prior to accessing the
dictionary of interned attributes.
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 173 +++++++++++++++++++++++++++++++++++++++++++---------------
attr.h | 2 +
common-main.c | 3 +
3 files changed, 133 insertions(+), 45 deletions(-)
@@ -23,28 +24,17 @@ static const char git_attr__unknown[] = "(builtin)unknown";#define ATTR__UNSET NULL#define ATTR__UNKNOWN git_attr__unknown-/* This is a randomly chosen prime. */-#define HASHSIZE 257-#ifndef DEBUG_ATTR#define DEBUG_ATTR 0#endif-/*-*NEEDSWORK:theglobaldictionaryoftheinternedattributes-*muststayasingletonevenafterwebecomethread-ready.-*Accesstothesemustbesurroundedwithmutexwhenithappens.-*/structgit_attr{-structgit_attr*next;-unsignedh;-intattr_nr;+intattr_nr;/* unique attribute number */intmaybe_macro;intmaybe_real;-charname[FLEX_ARRAY];+charname[FLEX_ARRAY];/* attribute name */};staticintattr_nr;-staticstructgit_attr*(git_attr_hash[HASHSIZE]);/**NEEDSWORK:maybe-real,maybe-macroarenotpropertyof
@@ -63,15 +53,94 @@ const char *git_attr_name(const struct git_attr *attr)returnattr->name;}-staticunsignedhash_name(constchar*name,intnamelen)+structattr_hashmap{+structhashmapmap;+#ifndef NO_PTHREADS+pthread_mutex_tmutex;+#endif+};++staticinlinevoidhashmap_lock(structattr_hashmap*map)+{+#ifndef NO_PTHREADS+pthread_mutex_lock(&map->mutex);+#endif+}++staticinlinevoidhashmap_unlock(structattr_hashmap*map){-unsignedval=0,c;+#ifndef NO_PTHREADS+pthread_mutex_unlock(&map->mutex);+#endif+}-while(namelen--){-c=*name++;-val=((val<<7)|(val>>22))^c;-}-returnval;+/*+*Theglobaldictionaryofallinternedattributes.This+*isasingletonobjectwhichissharedbetweenthreads.+*Accesstothisdictionarymustbesurroundedwithamutex.+*/+staticstructattr_hashmapg_attr_hashmap;++/* The container for objects stored in "struct attr_hashmap" */+structattr_hash_entry{+structhashmap_entryent;/* must be the first member! */+constchar*key;/* the key; memory should be owned by value */+size_tkeylen;/* length of the key */+void*value;/* the stored value */+};++/* attr_hashmap comparison function */+staticintattr_hash_entry_cmp(conststructattr_hash_entry*a,+conststructattr_hash_entry*b,+void*unused)+{+return(a->keylen!=b->keylen)||strncmp(a->key,b->key,a->keylen);+}++/* Initialize an 'attr_hashmap' object */+staticvoidattr_hashmap_init(structattr_hashmap*map)+{+hashmap_init(&map->map,(hashmap_cmp_fn)attr_hash_entry_cmp,0);+}++/*+*Retrievethe'value'storedinahashmapgiventheprovided'key'.+*Ifthereisnomatchingentry,returnNULL.+*/+staticvoid*attr_hashmap_get(structattr_hashmap*map,+constchar*key,size_tkeylen)+{+structattr_hash_entryk;+structattr_hash_entry*e;++if(!map->map.tablesize)+attr_hashmap_init(map);++hashmap_entry_init(&k,memhash(key,keylen));+k.key=key;+k.keylen=keylen;+e=hashmap_get(&map->map,&k,NULL);++returne?e->value:NULL;+}++/* Add 'value' to a hashmap based on the provided 'key'. */+staticvoidattr_hashmap_add(structattr_hashmap*map,+constchar*key,size_tkeylen,+void*value)+{+structattr_hash_entry*e;++if(!map->map.tablesize)+attr_hashmap_init(map);++e=xmalloc(sizeof(structattr_hash_entry));+hashmap_entry_init(e,memhash(key,keylen));+e->key=key;+e->keylen=keylen;+e->value=value;++hashmap_add(&map->map,e);}staticintattr_name_valid(constchar*name,size_tnamelen)
From: Brandon Williams <hidden> Date: 2017-01-23 20:36:23
Currently there is a reliance on 'check_all_attr' which is a global
array of 'attr_check_item' items which is used to store the value of
each attribute during the collection process.
This patch eliminates this global and instead creates an array per
'attr_check' instance which is then used in the attribute collection
process. This brings the attribute system one step closer to being
thread-safe.
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 114 +++++++++++++++++++++++++++++++++++++++++++----------------------
attr.h | 2 ++
2 files changed, 78 insertions(+), 38 deletions(-)
@@ -34,7 +34,6 @@ struct git_attr {intmaybe_real;charname[FLEX_ARRAY];/* attribute name */};-staticintattr_nr;/**NEEDSWORK:maybe-real,maybe-macroarenotpropertyof
@@ -45,9 +44,6 @@ static int attr_nr;*/staticintcannot_trust_maybe_real;-/* NEEDSWORK: This will become per git_attr_check */-staticstructattr_check_item*check_all_attr;-constchar*git_attr_name(conststructgit_attr*attr){returnattr->name;
@@ -791,16 +823,16 @@ static int path_matches(const char *pathname, int pathlen,pattern,prefix,pat->patternlen,pat->flags);}-staticintmacroexpand_one(intattr_nr,intrem);+staticintmacroexpand_one(structattr_check_item*all_attrs,intnr,intrem);-staticintfill_one(constchar*what,structmatch_attr*a,intrem)+staticintfill_one(constchar*what,structattr_check_item*all_attrs,+structmatch_attr*a,intrem){-structattr_check_item*check=check_all_attr;inti;-for(i=a->num_attr-1;0<rem&&0<=i;i--){+for(i=a->num_attr-1;rem>0&&i>=0;i--){structgit_attr*attr=a->state[i].attr;-constchar**n=&(check[attr->attr_nr].value);+constchar**n=&(all_attrs[attr->attr_nr].value);constchar*v=a->state[i].setto;if(*n==ATTR__UNKNOWN){
@@ -809,14 +841,15 @@ static int fill_one(const char *what, struct match_attr *a, int rem)attr,v);*n=v;rem--;-rem=macroexpand_one(attr->attr_nr,rem);+rem=macroexpand_one(all_attrs,attr->attr_nr,rem);}}returnrem;}staticintfill(constchar*path,intpathlen,intbasename_offset,-structattr_stack*stk,intrem)+structattr_stack*stk,structattr_check_item*all_attrs,+intrem){inti;constchar*base=stk->origin?stk->origin:"";
@@ -827,18 +860,18 @@ static int fill(const char *path, int pathlen, int basename_offset,continue;if(path_matches(path,pathlen,basename_offset,&a->u.pat,base,stk->originlen))-rem=fill_one("fill",a,rem);+rem=fill_one("fill",all_attrs,a,rem);}returnrem;}-staticintmacroexpand_one(intnr,intrem)+staticintmacroexpand_one(structattr_check_item*all_attrs,intnr,intrem){structattr_stack*stk;inti;-if(check_all_attr[nr].value!=ATTR__TRUE||-!check_all_attr[nr].attr->maybe_macro)+if(all_attrs[nr].value!=ATTR__TRUE||+!all_attrs[nr].attr->maybe_macro)returnrem;for(stk=attr_stack;stk;stk=stk->prev){
@@ -847,7 +880,7 @@ static int macroexpand_one(int nr, int rem)if(!ma->is_macro)continue;if(ma->u.attr->attr_nr==nr)-returnfill_one("expand",ma,rem);+returnfill_one("expand",all_attrs,ma,rem);}}
@@ -855,9 +888,9 @@ static int macroexpand_one(int nr, int rem)}/*-*Collectattributesforpathintothearraypointedtoby-*check_all_attr.Ifnumisnon-zero,onlyattributesincheck[]are-*collected.Otherwiseallattributesarecollected.+*Collectattributesforpathintothearraypointedtobycheck->all_attrs.+*Ifcheck->check_nrisnon-zero,onlyattributesincheck[]arecollected.+*Otherwiseallattributesarecollected.*/staticvoidcollect_some_attrs(constchar*path,structattr_check*check){
From: Brandon Williams <hidden> Date: 2017-01-23 20:36:26
From: Junio C Hamano <redacted>
This updates the other two ways the attribute check is done via an
array of "struct attr_check_item" elements. These two niches
appear only in "git check-attr".
* The caller does not know offhand what attributes it wants to ask
about and cannot use attr_check_initl() to prepare the
attr_check structure.
* The caller may not know what attributes it wants to ask at all,
and instead wants to learn everything that the given path has.
Such a caller can call attr_check_alloc() to allocate an empty
attr_check, and then call attr_check_append() to add attribute names
one by one.
Signed-off-by: Junio C Hamano <redacted>
Signed-off-by: Stefan Beller <redacted>
Signed-off-by: Brandon Williams <redacted>
---
attr.c | 38 ++++++++++++---------------------
attr.h | 9 +++-----
builtin/check-attr.c | 60 ++++++++++++++++++++++++++--------------------------
3 files changed, 47 insertions(+), 60 deletions(-)
@@ -837,42 +837,32 @@ int git_check_attrs(const char *path, int num, struct attr_check_item *check)return0;}-intgit_all_attrs(constchar*path,int*num,structattr_check_item**check)+voidgit_all_attrs(constchar*path,structattr_check*check){-inti,count,j;+inti;-collect_some_attrs(path,0,NULL);+attr_check_reset(check);+collect_some_attrs(path,check->check_nr,check->check);-/* Count the number of attributes that are set. */-count=0;-for(i=0;i<attr_nr;i++){-constchar*value=check_all_attr[i].value;-if(value!=ATTR__UNSET&&value!=ATTR__UNKNOWN)-++count;-}-*num=count;-ALLOC_ARRAY(*check,count);-j=0;for(i=0;i<attr_nr;i++){+constchar*name=check_all_attr[i].attr->name;constchar*value=check_all_attr[i].value;-if(value!=ATTR__UNSET&&value!=ATTR__UNKNOWN){-(*check)[j].attr=check_all_attr[i].attr;-(*check)[j].value=value;-++j;-}+structattr_check_item*item;+if(value==ATTR__UNSET||value==ATTR__UNKNOWN)+continue;+item=attr_check_append(check,git_attr(name));+item->value=value;}--return0;}-structattr_check*attr_check_alloc(void)+intgit_check_attr(constchar*path,structattr_check*check){-returnxcalloc(1,sizeof(structattr_check));+returngit_check_attrs(path,check->check_nr,check->check);}-intgit_check_attr(constchar*path,structattr_check*check)+structattr_check*attr_check_alloc(void){-returngit_check_attrs(path,check->check_nr,check->check);+returnxcalloc(1,sizeof(structattr_check));}structattr_check*attr_check_initl(constchar*one,...)