From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
The highlight of this patch series are that (1) peeled references are
not lost from the packed-refs file when a packed reference is deleted
and (2) there is more code sharing between the code used by "git
packed-refs" and the code used by repack_without_ref() for deleting a
packed reference. Along the way I have added a lot of documentation
and fixed several smaller bugs.
I had to add some sleeps in test t3210 but I hope that somebody can
find a way to eliminate them.
The last patch implements a change that I brushed off a long time ago
when Heiko suggested it. He was right (probably not for performance
reasons, but because it simplifies the code).
Summary:
Patches 1-4 document existing code.
Patches 5-6 random cleanup.
Patches 7-9 change the semantics of get_packed_ref() to make it more
reusable and change other locations to use this function.
Patch 10 extracts a function ref_resolves_to_object().
Patches 11-14 unify reference-peeling in two new functions,
peel_object() and peel_entry() (and fix an inconsistency in
peel_ref()).
Patch 15 introduces a new internal reference-iteration API and adjusts
some internal code to use it. The new API gives the callback function
direct access to the ref_entry. This corrects the handling of
current_ref during an iteration over only packed references.
Patches 16-17 add and fix a test of a spurious error message.
Patches 18-22 change the reference-deleting code to write peeled refs
to the packed-refs file. (Previously, whenever a packed reference was
deleted, all of the peeled values were lost when the packed-refs file
was rewritten.)
Patches 23-31 move the code from pack-refs.{c,h} into refs.{c,h},
adjust it to its new surroundings, and change it to share some code
with repack_without_ref().
Patches 32-33 simplify access to the main reference cache.
Michael Haggerty (33):
refs: document flags constants REF_*
refs: document the fields of struct ref_value
refs: document do_for_each_ref() and do_one_ref()
refs: document how current_ref is used
refs: define constant PEELED_LINE_LENGTH
do_for_each_ref_in_dirs(): remove dead code
get_packed_ref(): return a ref_entry
peel_ref(): use function get_packed_ref()
repack_without_ref(): use function get_packed_ref()
refs: extract a function ref_resolves_to_object()
refs: extract function peel_object()
peel_object(): give more specific information in return value
peel_ref(): fix return value for non-peelable, not-current reference
refs: extract a function peel_entry()
refs: change the internal reference-iteration API
t3210: test for spurious error messages for dangling packed refs
repack_without_ref(): silence errors for dangling packed refs
search_ref_dir(): return an index rather than a pointer
refs: change how packed refs are deleted
t3211: demonstrate loss of peeled refs if a packed ref is deleted
repack_without_ref(): write peeled refs in the rewritten file
refs: extract a function write_packed_entry()
pack-refs: rename handle_one_ref() to pack_one_ref()
pack-refs: merge code from pack-refs.{c,h} into refs.{c,h}
pack_one_ref(): rename "path" parameter to "refname"
refs: use same lock_file object for both ref-packing functions
pack_refs(): change to use do_for_each_entry()
refs: inline function do_not_prune()
pack_one_ref(): use function peel_entry()
pack_one_ref(): use write_packed_entry() to do the writing
pack_one_ref(): do some cheap tests before a more expensive one
refs: change do_for_each_*() functions to take ref_cache arguments
refs: handle the main ref_cache specially
Makefile | 2 -
builtin/clone.c | 1 -
builtin/pack-refs.c | 2 +-
pack-refs.c | 148 -----------
pack-refs.h | 18 --
refs.c | 699 ++++++++++++++++++++++++++++++++++++++-------------
refs.h | 35 +++
t/t3210-pack-refs.sh | 36 +++
t/t3211-peel-ref.sh | 9 +
9 files changed, 609 insertions(+), 341 deletions(-)
delete mode 100644 pack-refs.c
delete mode 100644 pack-refs.h
--
1.8.2.1
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
Instead of just returning a success/failure bit, return an enumeration
value that explains the reason for any failure. This will come in
handy shortly.
Signed-off-by: Michael Haggerty <redacted>
---
refs.c | 32 ++++++++++++++++++++++++--------
1 file changed, 24 insertions(+), 8 deletions(-)
@@ -1272,32 +1272,48 @@ static int filter_refs(const char *refname, const unsigned char *sha1, int flagsreturnfilter->fn(refname,sha1,flags,filter->cb_data);}+enumpeel_status{+/* object was peeled successfully: */+PEEL_PEELED=0,++/*+*objectcannotbepeeledbecausethenamedobject(oran+*objectreferredtobyataginthepeelchain),doesnot+*exist.+*/+PEEL_INVALID=-1,++/* object cannot be peeled because it is not a tag: */+PEEL_NON_TAG=-2,+};+/**Peelthenamedobject;i.e.,iftheobjectisatag,resolvethe-*tagrecursivelyuntilanon-tagisfound.Storetheresulttosha1-*andreturn0iffsuccessful.Iftheobjectisnotatagorisnot-*valid,return-1andleavesha1unchanged.+*tagrecursivelyuntilanon-tagisfound.Ifsuccessful,storethe+*resulttosha1andreturnPEEL_PEELED.Iftheobjectisnotatag+*orisnotvalid,returnPEEL_NON_TAGorPEEL_INVALID,respectively,+*andleavesha1unchanged.*/-staticintpeel_object(constunsignedchar*name,unsignedchar*sha1)+staticenumpeel_statuspeel_object(constunsignedchar*name,unsignedchar*sha1){structobject*o=lookup_unknown_object(name);if(o->type==OBJ_NONE){inttype=sha1_object_info(name,NULL);if(type<0)-return-1;+returnPEEL_INVALID;o->type=type;}if(o->type!=OBJ_TAG)-return-1;+returnPEEL_NON_TAG;o=deref_tag_noverify(o);if(!o)-return-1;+returnPEEL_INVALID;hashcpy(sha1,o->sha1);-return0;+returnPEEL_PEELED;}intpeel_ref(constchar*refname,unsignedchar*sha1)
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
Establish an internal API for iterating over references, which gives
the callback functions direct access to the ref_entry structure
describing the reference. (Do not change the iteration API that is
exposed outside of the module.)
Define a new internal callback signature
int each_ref_entry_fn(struct ref_entry *entry, void *cb_data)
Change do_for_each_ref_in_dir() and do_for_each_ref_in_dirs() to
accept each_ref_entry_fn callbacks, and rename them to
do_for_each_entry_in_dir() and do_for_each_entry_in_dirs(),
respectively. Adapt their callers accordingly.
Add a new function do_for_each_entry() analogous to do_for_each_ref()
but using the new callback style.
Change do_one_ref() into an each_ref_entry_fn that does some
bookkeeping and then calls a wrapped each_ref_fn.
Reimplement do_for_each_ref() in terms of do_for_each_entry(), using
do_one_ref() as an adapter.
Please note that the responsibility for setting current_ref remains in
do_one_ref(), which means that current_ref is *not* set when iterating
over references via the new internal API. This is not a disadvantage,
because current_ref is not needed by callers of the internal API (they
receive a pointer to the current ref_entry anyway). But more
importantly, this change prevents peel_ref() from returning invalid
results in the following scenario:
When iterating via the external API, the iteration always includes
both packed and loose references, and in particular never presents a
packed ref if there is a loose ref with the same name. The internal
API, on the other hand, gives the option to iterate over only the
packed references. During such an iteration, there is no check
whether the packed ref might be hidden by a loose ref of the same
name. But until now the packed ref was recorded in current_ref during
the iteration. So if peel_ref() were called with the reference name
corresponding to current ref, it would return the peeled version of
the packed ref even though there might be a loose ref that peels to a
different value. This scenario doesn't currently occur in the code,
but fix it to prevent things from breaking in a very confusing way in
the future.
Signed-off-by: Michael Haggerty <redacted>
---
refs.c | 142 +++++++++++++++++++++++++++++++++++++----------------------------
1 file changed, 81 insertions(+), 61 deletions(-)
@@ -556,22 +556,34 @@ static int ref_resolves_to_object(struct ref_entry *entry)*/staticstructref_entry*current_ref;+typedefinteach_ref_entry_fn(structref_entry*entry,void*cb_data);++structref_entry_cb{+constchar*base;+inttrim;+intflags;+each_ref_fn*fn;+void*cb_data;+};+/*-*Handleonereferenceinado_for_each_ref*()-styleiteration.+*Handleonereferenceinado_for_each_ref*()-styleiteration,+*callinganeach_ref_fnforeachentry.*/-staticintdo_one_ref(constchar*base,each_ref_fnfn,inttrim,-intflags,void*cb_data,structref_entry*entry)+staticintdo_one_ref(structref_entry*entry,void*cb_data){+structref_entry_cb*data=cb_data;intretval;-if(prefixcmp(entry->name,base))+if(prefixcmp(entry->name,data->base))return0;-if(!((flags&DO_FOR_EACH_INCLUDE_BROKEN)||+if(!((data->flags&DO_FOR_EACH_INCLUDE_BROKEN)||ref_resolves_to_object(entry)))return0;current_ref=entry;-retval=fn(entry->name+trim,entry->u.value.sha1,entry->flag,cb_data);+retval=data->fn(entry->name+data->trim,entry->u.value.sha1,+entry->flag,data->cb_data);current_ref=NULL;returnretval;}
@@ -580,11 +592,11 @@ static int do_one_ref(const char *base, each_ref_fn fn, int trim,*Callfnforeachreferenceindirthathasindexintherange*offset<=index<dir->nr.Recurseintosubdirectoriesthatarein*thatindexrange,sortingthembeforeiterating.Thisfunction-*doesnotsortdiritself;itshouldbesortedbeforehand.+*doesnotsortdiritself;itshouldbesortedbeforehand.fnis+*calledforallreferences,includingbrokenones.*/-staticintdo_for_each_ref_in_dir(structref_dir*dir,intoffset,-constchar*base,-each_ref_fnfn,inttrim,intflags,void*cb_data)+staticintdo_for_each_entry_in_dir(structref_dir*dir,intoffset,+each_ref_entry_fnfn,void*cb_data){inti;assert(dir->sorted==dir->nr);
@@ -594,10 +606,9 @@ static int do_for_each_ref_in_dir(struct ref_dir *dir, int offset,if(entry->flag&REF_DIR){structref_dir*subdir=get_ref_dir(entry);sort_ref_dir(subdir);-retval=do_for_each_ref_in_dir(subdir,0,-base,fn,trim,flags,cb_data);+retval=do_for_each_entry_in_dir(subdir,0,fn,cb_data);}else{-retval=do_one_ref(base,fn,trim,flags,cb_data,entry);+retval=fn(entry,cb_data);}if(retval)returnretval;
@@ -610,12 +621,12 @@ static int do_for_each_ref_in_dir(struct ref_dir *dir, int offset,*byrefname.Recurseintosubdirectories.Ifavalueentryappears*inbothdir1anddir2,thenonlyprocesstheversionthatisin*dir2.Theinputdirsmustalreadybesorted,butsubdirswillbe-*sortedasneeded.+*sortedasneeded.fniscalledforallreferences,including+*brokenones.*/-staticintdo_for_each_ref_in_dirs(structref_dir*dir1,-structref_dir*dir2,-constchar*base,each_ref_fnfn,inttrim,-intflags,void*cb_data)+staticintdo_for_each_entry_in_dirs(structref_dir*dir1,+structref_dir*dir2,+each_ref_entry_fnfn,void*cb_data){intretval;inti1=0,i2=0;
@@ -626,12 +637,10 @@ static int do_for_each_ref_in_dirs(struct ref_dir *dir1,structref_entry*e1,*e2;intcmp;if(i1==dir1->nr){-returndo_for_each_ref_in_dir(dir2,i2,-base,fn,trim,flags,cb_data);+returndo_for_each_entry_in_dir(dir2,i2,fn,cb_data);}if(i2==dir2->nr){-returndo_for_each_ref_in_dir(dir1,i1,-base,fn,trim,flags,cb_data);+returndo_for_each_entry_in_dir(dir1,i1,fn,cb_data);}e1=dir1->entries[i1];e2=dir2->entries[i2];
@@ -643,14 +652,13 @@ static int do_for_each_ref_in_dirs(struct ref_dir *dir1,structref_dir*subdir2=get_ref_dir(e2);sort_ref_dir(subdir1);sort_ref_dir(subdir2);-retval=do_for_each_ref_in_dirs(-subdir1,subdir2,-base,fn,trim,flags,cb_data);+retval=do_for_each_entry_in_dirs(+subdir1,subdir2,fn,cb_data);i1++;i2++;}elseif(!(e1->flag&REF_DIR)&&!(e2->flag&REF_DIR)){/* Both are references; ignore the one from dir1. */-retval=do_one_ref(base,fn,trim,flags,cb_data,e2);+retval=fn(e2,cb_data);i1++;i2++;}else{
@@ -669,11 +677,10 @@ static int do_for_each_ref_in_dirs(struct ref_dir *dir1,if(e->flag&REF_DIR){structref_dir*subdir=get_ref_dir(e);sort_ref_dir(subdir);-retval=do_for_each_ref_in_dir(-subdir,0,-base,fn,trim,flags,cb_data);+retval=do_for_each_entry_in_dir(+subdir,0,fn,cb_data);}else{-retval=do_one_ref(base,fn,trim,flags,cb_data,e);+retval=fn(e,cb_data);}}if(retval)
@@ -1870,20 +1889,21 @@ struct repack_without_ref_sb {intfd;};-staticintrepack_without_ref_fn(constchar*refname,constunsignedchar*sha1,-intflags,void*cb_data)+staticintrepack_without_ref_fn(structref_entry*entry,void*cb_data){structrepack_without_ref_sb*data=cb_data;charline[PATH_MAX+100];intlen;-if(!strcmp(data->refname,refname))+if(!strcmp(data->refname,entry->name))return0;+if(!ref_resolves_to_object(entry))+return0;/* Skip broken refs */len=snprintf(line,sizeof(line),"%s %s\n",-sha1_to_hex(sha1),refname);+sha1_to_hex(entry->u.value.sha1),entry->name);/* this should not happen but just being defensive */if(len>sizeof(line))-die("too long a refname '%s'",refname);+die("too long a refname '%s'",entry->name);write_or_die(data->fd,line,len);return0;}
@@ -1907,7 +1927,7 @@ static int repack_without_ref(const char *refname)}clear_packed_ref_cache(refs);packed=get_packed_refs(refs);-do_for_each_ref_in_dir(packed,0,"",repack_without_ref_fn,0,0,&data);+do_for_each_entry_in_dir(packed,0,repack_without_ref_fn,&data);returncommit_lock_file(&packlock);}
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
Peel the entry, and as a side effect store the peeled value in the
entry. Use this function from two places in peel_ref(); a third
caller will be added soon.
Please note that this change can lead to ref_entries for unpacked refs
being peeled. This has no practical benefit but is harmless.
Signed-off-by: Michael Haggerty <redacted>
---
refs.c | 61 ++++++++++++++++++++++++++++++++++++++++++++++++-------------
1 file changed, 48 insertions(+), 13 deletions(-)
@@ -1286,6 +1286,16 @@ enum peel_status {/* object cannot be peeled because it is not a tag: */PEEL_NON_TAG=-2,++/* ref_entry contains no peeled value because it is a symref: */+PEEL_IS_SYMREF=-3,++/*+*ref_entrycannotbepeeledbecauseitisbroken(i.e.,the+*symbolicreferencecannotevenberesolvedtoanobject+*name):+*/+PEEL_BROKEN=-4,};/*
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
Stop emitting an error message for dangling packed references found
when deleting another packed reference. See the previous commit for a
longer explanation of the issue.
Change repack_without_ref_fn() to silently ignore dangling packed
references.
Signed-off-by: Michael Haggerty <redacted>
---
refs.c | 17 ++++++++++-------
t/t3210-pack-refs.sh | 2 +-
2 files changed, 11 insertions(+), 8 deletions(-)
@@ -531,15 +531,17 @@ static void sort_ref_dir(struct ref_dir *dir)/**Returntrueiffthereferencedescribedbyentrycanberesolvedto-*anobjectinthedatabase.Emitawarningifthereferred-to-*objectdoesnotexist.+*anobjectinthedatabase.Ifreport_errorsistrue,emita+*warningifthereferred-toobjectdoesnotexist.*/-staticintref_resolves_to_object(structref_entry*entry)+staticintref_resolves_to_object(structref_entry*entry,intreport_errors){if(entry->flag&REF_ISBROKEN)return0;if(!has_sha1_file(entry->u.value.sha1)){-error("%s does not point to a valid object!",entry->name);+if(report_errors)+error("%s does not point to a valid object!",+entry->name);return0;}return1;
@@ -1897,8 +1899,9 @@ static int repack_without_ref_fn(struct ref_entry *entry, void *cb_data)if(!strcmp(data->refname,entry->name))return0;-if(!ref_resolves_to_object(entry))-return0;/* Skip broken refs */+/* Silently skip broken refs: */+if(!ref_resolves_to_object(entry,0))+return0;len=snprintf(line,sizeof(line),"%s %s\n",sha1_to_hex(entry->u.value.sha1),entry->name);/* this should not happen but just being defensive */
@@ -142,7 +142,7 @@ test_expect_success 'delete ref with dangling packed version' 'test_cmp/dev/nullresult'-test_expect_failure'delete ref while another dangling packed ref''+test_expect_success'delete ref while another dangling packed ref''gitbranchlamb&&gitcommit--allow-empty-m"future garbage"&&gitpack-refs--all&&
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
Add a function remove_ref(), which removes a single entry from a
reference cache.
Use this function to reimplement repack_without_ref(). The old
version iterated over all refs, packing all of them except for the one
to be deleted, then discarded the entire packed reference cache. The
new version deletes the doomed reference from the cache *before*
iterating.
This has two advantages:
* the code for writing packed-refs becomes simpler, because it doesn't
have to exclude one of the references.
* it is no longer necessary to discard the packed refs cache after
deleting a reference: symbolic refs cannot be packed, so packed
references cannot depend on each other, so the rest of the packed
refs cache remains valid after a reference is deleted.
Signed-off-by: Michael Haggerty <redacted>
---
refs.c | 84 +++++++++++++++++++++++++++++++++++++++++++++++++++++-------------
1 file changed, 68 insertions(+), 16 deletions(-)
@@ -1913,7 +1957,7 @@ static int repack_without_ref_fn(struct ref_entry *entry, void *cb_data)/* this should not happen but just being defensive */if(len>sizeof(line))die("too long a refname '%s'",entry->name);-write_or_die(data->fd,line,len);+write_or_die(*fd,line,len);return0;}
@@ -1921,22 +1965,30 @@ static struct lock_file packlock;staticintrepack_without_ref(constchar*refname){-structrepack_without_ref_sbdata;+intfd;structref_cache*refs=get_ref_cache(NULL);structref_dir*packed;if(!get_packed_ref(refname))return0;/* refname does not exist in packed refs */-data.refname=refname;-data.fd=hold_lock_file_for_update(&packlock,git_path("packed-refs"),0);-if(data.fd<0){+fd=hold_lock_file_for_update(&packlock,git_path("packed-refs"),0);+if(fd<0){unable_to_lock_error(git_path("packed-refs"),errno);returnerror("cannot delete '%s' from packed refs",refname);}clear_packed_ref_cache(refs);packed=get_packed_refs(refs);-do_for_each_entry_in_dir(packed,0,repack_without_ref_fn,&data);+/* Remove refname from the cache. */+if(remove_entry(packed,refname)==-1){+/*+*Thepackedentrydisappearedwhilewewere+*acquiringthelock.+*/+rollback_lock_file(&packlock);+return0;+}+do_for_each_entry_in_dir(packed,0,repack_ref_fn,&fd);returncommit_lock_file(&packlock);}
@@ -1965,7 +2017,7 @@ int delete_ref(const char *refname, const unsigned char *sha1, int delopt)ret|=repack_without_ref(lock->ref_name);unlink_or_warn(git_path("logs/%s",lock->ref_name));-invalidate_ref_cache(NULL);+clear_loose_ref_cache(get_ref_cache(NULL));unlock_ref(lock);returnret;}
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
Extract the I/O code from the "business logic" in repack_ref_fn().
Later there will be another caller for this function.
Signed-off-by: Michael Haggerty <redacted>
---
refs.c | 40 +++++++++++++++++++++++++++-------------
1 file changed, 27 insertions(+), 13 deletions(-)
@@ -1956,29 +1956,43 @@ struct ref_lock *lock_any_ref_for_update(const char *refname,returnlock_ref_sha1_basic(refname,old_sha1,flags,NULL);}-staticintrepack_ref_fn(structref_entry*entry,void*cb_data)+/*+*Writeanentrytothepacked-refsfileforthespecifiedrefname.+*Ifpeeledisnon-NULL,writeitastheentry'speeledvalue.+*/+staticvoidwrite_packed_entry(intfd,char*refname,unsignedchar*sha1,+unsignedchar*peeled){-int*fd=cb_data;charline[PATH_MAX+100];intlen;-/* Silently skip broken refs: */-if(!ref_resolves_to_object(entry,0))-return0;len=snprintf(line,sizeof(line),"%s %s\n",-sha1_to_hex(entry->u.value.sha1),entry->name);+sha1_to_hex(sha1),refname);/* this should not happen but just being defensive */if(len>sizeof(line))-die("too long a refname '%s'",entry->name);-write_or_die(*fd,line,len);-if(!peel_entry(entry)){-/* This reference could be peeled; write the peeled value: */+die("too long a refname '%s'",refname);+write_or_die(fd,line,len);++if(peeled){if(snprintf(line,sizeof(line),"^%s\n",-sha1_to_hex(entry->u.value.peeled))!=-PEELED_LINE_LENGTH)+sha1_to_hex(peeled))!=PEELED_LINE_LENGTH)die("internal error");-write_or_die(*fd,line,PEELED_LINE_LENGTH);+write_or_die(fd,line,PEELED_LINE_LENGTH);}+}++staticintrepack_ref_fn(structref_entry*entry,void*cb_data)+{+int*fd=cb_data;+enumpeel_statuspeel_status;++/* Silently skip broken refs: */+if(!ref_resolves_to_object(entry,0))+return0;+peel_status=peel_entry(entry);+write_packed_entry(*fd,entry->name,entry->u.value.sha1,+peel_status==PEEL_PEELED?+entry->u.value.peeled:NULL);return0;}
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
Add a test that demonstrates that the peeled values recorded in
packed-refs are lost if a packed ref is deleted. (The code in
repack_without_ref() doesn't even attempt to write peeled refs.) This
will be fixed in a moment.
Signed-off-by: Michael Haggerty <redacted>
---
t/t3211-peel-ref.sh | 9 +++++++++
1 file changed, 9 insertions(+)
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
When a reference that existed in the packed-refs file is deleted, the
packed-refs file must be rewritten. Previously, the file was
rewritten without any peeled refs, even if the file contained peeled
refs when it was read. This was not a bug, because the packed-refs
file header didn't claim that the file contained peeled values. But
it had a performance cost, because the repository would lose the
benefit of having precomputed peeled references until pack-refs was
run again.
Teach repack_without_ref() to write peeled refs to the packed-refs
file (regardless of whether they were present in the old version of
the file).
This means that if the old version of the packed-refs file was not
fully peeled, then repack_without_ref() will have to peel references.
To avoid the expense of reading lots of loose references, we take two
shortcuts relative to pack-refs:
* If the peeled value of a reference is already known (i.e., because
it was read from the old version of the packed-refs file), then
output that peeled value again without any checks. This is the
usual code path and should avoid any noticeable overhead. (This is
different than pack-refs, which always re-peels references.)
* We don't verify that the packed ref is still current. It could be
that a packed references is overridden by a loose reference, in
which case the packed ref is no longer needed and might even refer
to an object that has been garbage collected. But we don't check;
instead, we just try to peel all references. If peeling is
successful, the peeled value is written out (even though it might
not be needed any more); if not, then the reference is silently
omitted from the output.
The extra overhead of peeling references in repack_without_ref()
should only be incurred the first time the packed-refs file is written
by a version of Git that knows about the "fully-peeled" attribute.
Signed-off-by: Michael Haggerty <redacted>
---
This change could cause a noticeable delay on deleting a reference if
it causes a non-fully-peeled packed-refs file to be fully peeled. But
after the file has been fully peeled once, deleting a reference should
be as fast as before. In exchange, reference advertising should be
faster after this change because peeled values will not be lost
whenever a packed reference is deleted.
refs.c | 23 +++++++++++++++++++++++
t/t3211-peel-ref.sh | 2 +-
2 files changed, 24 insertions(+), 1 deletion(-)
@@ -1958,6 +1971,15 @@ static int repack_ref_fn(struct ref_entry *entry, void *cb_data)if(len>sizeof(line))die("too long a refname '%s'",entry->name);write_or_die(*fd,line,len);+if(!peel_entry(entry)){+/* This reference could be peeled; write the peeled value: */+if(snprintf(line,sizeof(line),"^%s\n",+sha1_to_hex(entry->u.value.peeled))!=+PEELED_LINE_LENGTH)+die("internal error");+write_or_die(*fd,line,PEELED_LINE_LENGTH);+}+return0;}
@@ -1988,6 +2010,7 @@ static int repack_without_ref(const char *refname)rollback_lock_file(&packlock);return0;}+write_or_die(fd,PACKED_REFS_HEADER,strlen(PACKED_REFS_HEADER));do_for_each_entry_in_dir(packed,0,repack_ref_fn,&fd);returncommit_lock_file(&packlock);}
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
Make this function conform to the naming convention established in
65385ef7d4 for the rest of the refs.c file.
Signed-off-by: Michael Haggerty <redacted>
---
refs.c | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
@@ -2001,7 +2001,7 @@ static int do_not_prune(int flags)return(flags&(REF_ISSYMREF|REF_ISPACKED));}-staticintpack_one_ref(constchar*path,constunsignedchar*sha1,+staticintpack_one_ref(constchar*refname,constunsignedchar*sha1,intflags,void*cb_data){structpack_refs_cb_data*cb=cb_data;
@@ -2011,27 +2011,27 @@ static int pack_one_ref(const char *path, const unsigned char *sha1,/* Do not pack the symbolic refs */if((flags&REF_ISSYMREF))return0;-is_tag_ref=!prefixcmp(path,"refs/tags/");+is_tag_ref=!prefixcmp(refname,"refs/tags/");/* ALWAYS pack refs that were already packed or are tags */if(!(cb->flags&PACK_REFS_ALL)&&!is_tag_ref&&!(flags&REF_ISPACKED))return0;-fprintf(cb->refs_file,"%s %s\n",sha1_to_hex(sha1),path);+fprintf(cb->refs_file,"%s %s\n",sha1_to_hex(sha1),refname);-o=parse_object_or_die(sha1,path);+o=parse_object_or_die(sha1,refname);if(o->type==OBJ_TAG){-o=deref_tag(o,path,0);+o=deref_tag(o,refname,0);if(o)fprintf(cb->refs_file,"^%s\n",sha1_to_hex(o->sha1));}if((cb->flags&PACK_REFS_PRUNE)&&!do_not_prune(flags)){-intnamelen=strlen(path)+1;+intnamelen=strlen(refname)+1;structref_to_prune*n=xcalloc(1,sizeof(*n)+namelen);hashcpy(n->sha1,sha1);-strcpy(n->name,path);+strcpy(n->name,refname);n->next=cb->ref_to_prune;cb->ref_to_prune=n;}
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
Use a single struct lock_file for both pack_refs() and
repack_without_ref().
Signed-off-by: Michael Haggerty <redacted>
---
refs.c | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
@@ -2099,7 +2099,7 @@ int pack_refs(unsigned int flags)memset(&cbdata,0,sizeof(cbdata));cbdata.flags=flags;-fd=hold_lock_file_for_update(&packed,git_path("packed-refs"),+fd=hold_lock_file_for_update(&packlock,git_path("packed-refs"),LOCK_DIE_ON_ERROR);cbdata.refs_file=fdopen(fd,"w");if(!cbdata.refs_file)
@@ -2118,8 +2118,8 @@ int pack_refs(unsigned int flags)*assign-1tothelockfiledescriptorsothatcommit_lock_file()*won'ttrytoclose()it.*/-packed.fd=-1;-if(commit_lock_file(&packed)<0)+packlock.fd=-1;+if(commit_lock_file(&packlock)<0)die_errno("unable to overwrite old ref-pack file");prune_refs(cbdata.ref_to_prune);return0;
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
Function do_not_prune() was redundantly checking REF_ISSYMREF, which
was already tested at the top of pack_one_ref(), so remove that check.
And the rest was trivial, so inline the function.
Signed-off-by: Michael Haggerty <redacted>
---
refs.c | 11 ++---------
1 file changed, 2 insertions(+), 9 deletions(-)
@@ -1993,14 +1993,6 @@ struct pack_refs_cb_data {FILE*refs_file;};-staticintdo_not_prune(intflags)-{-/* If it is already packed or if it is a symref,-*donotpruneit.-*/-return(flags&(REF_ISSYMREF|REF_ISPACKED));-}-staticintpack_one_ref(structref_entry*entry,void*cb_data){structpack_refs_cb_data*cb=cb_data;
@@ -2028,7 +2020,8 @@ static int pack_one_ref(struct ref_entry *entry, void *cb_data)sha1_to_hex(o->sha1));}-if((cb->flags&PACK_REFS_PRUNE)&&!do_not_prune(entry->flag)){+/* If the ref was already packed, there is no need to prune it. */+if((cb->flags&PACK_REFS_PRUNE)&&!(entry->flag&REF_ISPACKED)){intnamelen=strlen(entry->name)+1;structref_to_prune*n=xcalloc(1,sizeof(*n)+namelen);hashcpy(n->sha1,entry->u.value.sha1);
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
Change pack_refs() to work with a file descriptor instead of a FILE*
(making the file-locking code less awkward) and use
write_packed_entry() to do the writing.
Signed-off-by: Michael Haggerty <redacted>
---
Please checked that I am using the file-locking API correctly; I'm not
that familiar with it.
refs.c | 33 ++++++++-------------------------
1 file changed, 8 insertions(+), 25 deletions(-)
@@ -2017,15 +2017,13 @@ static int pack_one_ref(struct ref_entry *entry, void *cb_data)!(entry->flag&REF_ISPACKED))return0;-fprintf(cb->refs_file,"%s %s\n",sha1_to_hex(entry->u.value.sha1),-entry->name);-peel_status=peel_entry(entry,1);-if(peel_status==PEEL_PEELED)-fprintf(cb->refs_file,"^%s\n",sha1_to_hex(entry->u.value.peeled));-elseif(peel_status!=PEEL_NON_TAG)+if(peel_status!=PEEL_PEELED&&peel_status!=PEEL_NON_TAG)die("internal error peeling reference %s (%s)",entry->name,sha1_to_hex(entry->u.value.sha1));+write_packed_entry(cb->fd,entry->name,entry->u.value.sha1,+peel_status==PEEL_PEELED?+entry->u.value.peeled:NULL);/* If the ref was already packed, there is no need to prune it. */if((cb->flags&PACK_REFS_PRUNE)&&!(entry->flag&REF_ISPACKED)){
@@ -2094,32 +2092,17 @@ static struct lock_file packlock;intpack_refs(unsignedintflags){-intfd;structpack_refs_cb_datacbdata;memset(&cbdata,0,sizeof(cbdata));cbdata.flags=flags;-fd=hold_lock_file_for_update(&packlock,git_path("packed-refs"),-LOCK_DIE_ON_ERROR);-cbdata.refs_file=fdopen(fd,"w");-if(!cbdata.refs_file)-die_errno("unable to create ref-pack file structure");+cbdata.fd=hold_lock_file_for_update(&packlock,git_path("packed-refs"),+LOCK_DIE_ON_ERROR);-/* perhaps other traits later as well */-fprintf(cbdata.refs_file,"# pack-refs with: peeled fully-peeled \n");+write_or_die(cbdata.fd,PACKED_REFS_HEADER,strlen(PACKED_REFS_HEADER));do_for_each_entry(NULL,"",pack_one_ref,&cbdata);-if(ferror(cbdata.refs_file))-die("failed to write ref-pack file");-if(fflush(cbdata.refs_file)||fsync(fd)||fclose(cbdata.refs_file))-die_errno("failed to write ref-pack file");-/*-*Sincethelockfilewasfdopen()'edandthenfclose()'edabove,-*assign-1tothelockfiledescriptorsothatcommit_lock_file()-*won'ttrytoclose()it.-*/-packlock.fd=-1;if(commit_lock_file(&packlock)<0)die_errno("unable to overwrite old ref-pack file");prune_refs(cbdata.ref_to_prune);
@@ -2101,7 +2100,7 @@ int pack_refs(unsigned int flags)write_or_die(cbdata.fd,PACKED_REFS_HEADER,strlen(PACKED_REFS_HEADER));-do_for_each_entry(NULL,"",pack_one_ref,&cbdata);+do_for_each_entry(get_ref_cache(NULL),"",pack_one_ref,&cbdata);if(commit_lock_file(&packlock)<0)die_errno("unable to overwrite old ref-pack file");prune_refs(cbdata.ref_to_prune);
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
Hold the ref_cache instance for the main repository in a dedicated,
statically-allocated instance to avoid the need for a function call
and a linked-list traversal when it is needed.
Suggested by: Heiko Voigt [off-list ref]
Signed-off-by: Michael Haggerty <redacted>
---
refs.c | 60 +++++++++++++++++++++++++++++++-----------------------------
1 file changed, 31 insertions(+), 29 deletions(-)
@@ -812,9 +812,13 @@ static struct ref_cache {structref_cache*next;structref_entry*loose;structref_entry*packed;-/* The submodule name, or "" for the main repo. */-charname[FLEX_ARRAY];-}*ref_cache;+/*+*Thesubmodulename,or""forthemainrepo.Weallocate+*length1ratherthanFLEX_ARRAYsothatthemainref_cache+*isinitializedcorrectly.+*/+charname[1];+}ref_cache,*submodule_ref_caches;staticvoidclear_packed_ref_cache(structref_cache*refs){
@@ -2100,7 +2104,7 @@ int pack_refs(unsigned int flags)write_or_die(cbdata.fd,PACKED_REFS_HEADER,strlen(PACKED_REFS_HEADER));-do_for_each_entry(get_ref_cache(NULL),"",pack_one_ref,&cbdata);+do_for_each_entry(&ref_cache,"",pack_one_ref,&cbdata);if(commit_lock_file(&packlock)<0)die_errno("unable to overwrite old ref-pack file");prune_refs(cbdata.ref_to_prune);
@@ -2137,8 +2140,8 @@ static int repack_without_ref(const char *refname)unable_to_lock_error(git_path("packed-refs"),errno);returnerror("cannot delete '%s' from packed refs",refname);}-clear_packed_ref_cache(refs);-packed=get_packed_refs(refs);+clear_packed_ref_cache(&ref_cache);+packed=get_packed_refs(&ref_cache);/* Remove refname from the cache. */if(remove_entry(packed,refname)==-1){/*
@@ -2178,7 +2181,7 @@ int delete_ref(const char *refname, const unsigned char *sha1, int delopt)ret|=repack_without_ref(lock->ref_name);unlink_or_warn(git_path("logs/%s",lock->ref_name));-clear_loose_ref_cache(get_ref_cache(NULL));+clear_loose_ref_cache(&ref_cache);unlock_ref(lock);returnret;}
@@ -2200,7 +2203,6 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logmsstructstatloginfo;intlog=!lstat(git_path("logs/%s",oldrefname),&loginfo);constchar*symref=NULL;-structref_cache*refs=get_ref_cache(NULL);if(log&&S_ISLNK(loginfo.st_mode))returnerror("reflog for %s is a symlink",oldrefname);
@@ -2212,10 +2214,10 @@ int rename_ref(const char *oldrefname, const char *newrefname, const char *logmsif(!symref)returnerror("refname %s not found",oldrefname);-if(!is_refname_available(newrefname,oldrefname,get_packed_refs(refs)))+if(!is_refname_available(newrefname,oldrefname,get_packed_refs(&ref_cache)))return1;-if(!is_refname_available(newrefname,oldrefname,get_loose_refs(refs)))+if(!is_refname_available(newrefname,oldrefname,get_loose_refs(&ref_cache)))return1;if(log&&rename(git_path("logs/%s",oldrefname),git_path(TMP_RENAMED_LOG)))
@@ -2471,7 +2473,7 @@ int write_ref_sha1(struct ref_lock *lock,unlock_ref(lock);return-1;}-clear_loose_ref_cache(get_ref_cache(NULL));+clear_loose_ref_cache(&ref_cache);if(log_ref_write(lock->ref_name,lock->old_sha1,sha1,logmsg)<0||(strcmp(lock->ref_name,lock->orig_ref_name)&&log_ref_write(lock->orig_ref_name,lock->old_sha1,sha1,logmsg)<0)){
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
This code is about to be moved, so name the function more
distinctively.
Signed-off-by: Michael Haggerty <redacted>
---
pack-refs.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
@@ -23,7 +23,7 @@ static int do_not_prune(int flags)return(flags&(REF_ISSYMREF|REF_ISPACKED));}-staticinthandle_one_ref(constchar*path,constunsignedchar*sha1,+staticintpack_one_ref(constchar*path,constunsignedchar*sha1,intflags,void*cb_data){structpack_refs_cb_data*cb=cb_data;
@@ -130,7 +130,7 @@ int pack_refs(unsigned int flags)/* perhaps other traits later as well */fprintf(cbdata.refs_file,"# pack-refs with: peeled fully-peeled \n");-for_each_ref(handle_one_ref,&cbdata);+for_each_ref(pack_one_ref,&cbdata);if(ferror(cbdata.refs_file))die("failed to write ref-pack file");if(fflush(cbdata.refs_file)||fsync(fd)||fclose(cbdata.refs_file))
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
Change pack_one_ref() to call peel_entry() rather than using its own
code for peeling references. Aside from sharing code, this lets it
take advantage of the optimization introduced by 6c4a060d7d.
Please note that we *could* use any peeled values that happen to
already be stored in the ref_entries, which would avoid some object
lookups for references that were already packed. But doing so would
also propagate any peeling errors across runs of "git pack-refs" and
give no way to recover from such errors. And "git pack-refs" isn't
run often enough that the performance cost is a problem. So instead,
add a new option to peel_entry() to force the entry to be re-peeled,
and call it with that option set.
Signed-off-by: Michael Haggerty <redacted>
---
I am not so familiar with the object DB API so it would be good for
somebody to verify that the slightly different method that pack-refs
now uses to peel references is equivalent to the old method.
Please note that this version of the patch series (in contrast to one
I shared before via github) re-peels all references when pack-refs is
run rather than reusing any available peeled values.
refs.c | 39 +++++++++++++++++++++++----------------
1 file changed, 23 insertions(+), 16 deletions(-)
@@ -1996,7 +2004,7 @@ struct pack_refs_cb_data {staticintpack_one_ref(structref_entry*entry,void*cb_data){structpack_refs_cb_data*cb=cb_data;-structobject*o;+enumpeel_statuspeel_status;intis_tag_ref;/* Do not pack symbolic or broken refs: */
@@ -2012,13 +2020,12 @@ static int pack_one_ref(struct ref_entry *entry, void *cb_data)fprintf(cb->refs_file,"%s %s\n",sha1_to_hex(entry->u.value.sha1),entry->name);-o=parse_object_or_die(entry->u.value.sha1,entry->name);-if(o->type==OBJ_TAG){-o=deref_tag(o,entry->name,0);-if(o)-fprintf(cb->refs_file,"^%s\n",-sha1_to_hex(o->sha1));-}+peel_status=peel_entry(entry,1);+if(peel_status==PEEL_PEELED)+fprintf(cb->refs_file,"^%s\n",sha1_to_hex(entry->u.value.peeled));+elseif(peel_status!=PEEL_NON_TAG)+die("internal error peeling reference %s (%s)",+entry->name,sha1_to_hex(entry->u.value.sha1));/* If the ref was already packed, there is no need to prune it. */if((cb->flags&PACK_REFS_PRUNE)&&!(entry->flag&REF_ISPACKED)){
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
Instead of copying the reference's SHA1 into a caller-supplied
variable, just return the ref_entry itself (or NULL if there is no
such entry). This change will allow the function to be used from
elsewhere.
Signed-off-by: Michael Haggerty <redacted>
---
refs.c | 20 +++++++++-----------
1 file changed, 9 insertions(+), 11 deletions(-)
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
Change search_ref_dir() to return the index of the sought entry (or -1
on error) rather than a pointer to the entry. This will make it more
natural to use the function for removing an entry from the list.
Signed-off-by: Michael Haggerty <redacted>
---
refs.c | 30 ++++++++++++++++++------------
1 file changed, 18 insertions(+), 12 deletions(-)
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
The old version was inconsistent: when a reference was
REF_KNOWS_PEELED but with a null peeled value, it returned non-zero
for the current reference but zero for other references. Change the
behavior for non-current references to match that of current_ref,
which is what callers expect. Document the behavior.
Current callers did not trigger the previously-buggy behavior.
Signed-off-by: Michael Haggerty <redacted>
---
refs.c | 5 ++++-
refs.h | 8 ++++++++
2 files changed, 12 insertions(+), 1 deletion(-)
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
It is a nice, logical unit of work, and putting it in a function
removes the need to use a goto in peel_ref(). Soon it will also have
other uses.
The algorithm is unchanged.
Signed-off-by: Michael Haggerty <redacted>
---
refs.c | 50 ++++++++++++++++++++++++++++++--------------------
1 file changed, 30 insertions(+), 20 deletions(-)
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
There is no way to drop out of the while loop. This code has been
dead since 432ad41e.
Signed-off-by: Michael Haggerty <redacted>
---
refs.c | 7 -------
1 file changed, 7 deletions(-)
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
Document the bits that can appear in the "flags" parameter passed to
an each_ref_function and/or in the ref_entry::flag field.
Signed-off-by: Michael Haggerty <redacted>
---
refs.c | 12 +++++++++++-
refs.h | 13 +++++++++++++
2 files changed, 24 insertions(+), 1 deletion(-)
@@ -158,7 +158,17 @@ struct ref_dir {structref_entry**entries;};-/* ISSYMREF=0x01, ISPACKED=0x02, and ISBROKEN=0x04 are public interfaces */+/*+*Bitvaluesforref_entry::flag.REF_ISSYMREF=0x01,+*REF_ISPACKED=0x02,andREF_ISBROKEN=0x04arepublicvalues;see+*refs.h.+*/++/*+*Thefieldref_entry->u.value.peeledofthisvalueentrycontains+*thecorrectpeeledvalueforthereference,whichmightbe+*null_sha1ifthereferenceisnotatagorifitisbroken.+*/#define REF_KNOWS_PEELED 0x08/* ref_entry represents a directory of references */
@@ -10,8 +10,21 @@ struct ref_lock {intforce_write;};+/*+*Bitvaluessetintheflagsargumentpassedtoeach_ref_fn():+*/++/* Reference is a symbolic reference. */#define REF_ISSYMREF 0x01++/* Reference is a packed reference. */#define REF_ISPACKED 0x02++/*+*Referencecannotberesolvedtoanobjectname:danglingsymbolic+*reference(directlyorindirectly),corruptreferencefile,or+*symbolicreferencereferstoill-formattedreferencename.+*/#define REF_ISBROKEN 0x04/*
@@ -805,6 +805,9 @@ void invalidate_ref_cache(const char *submodule)clear_loose_ref_cache(refs);}+/* The length of a peeled reference line in packed-refs, including EOL: */+#define PEELED_LINE_LENGTH 42+/**Parseonelinefromapacked-refsfile.WritetheSHA1tosha1.*Returnapointertotherefnamewithintheline(null-terminated),
@@ -1818,9 +1818,11 @@ static int repack_without_ref(const char *refname){structrepack_without_ref_sbdata;structref_cache*refs=get_ref_cache(NULL);-structref_dir*packed=get_packed_refs(refs);-if(find_ref(packed,refname)==NULL)-return0;+structref_dir*packed;++if(!get_packed_ref(refname))+return0;/* refname does not exist in packed refs */+data.refname=refname;data.fd=hold_lock_file_for_update(&packlock,git_path("packed-refs"),0);if(data.fd<0){
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
It is a nice unit of work and soon will be needed from multiple
locations.
Signed-off-by: Michael Haggerty <redacted>
---
refs.c | 28 ++++++++++++++++++++--------
1 file changed, 20 insertions(+), 8 deletions(-)
@@ -529,6 +529,22 @@ static void sort_ref_dir(struct ref_dir *dir)#define DO_FOR_EACH_INCLUDE_BROKEN 0x01/*+*Returntrueiffthereferencedescribedbyentrycanberesolvedto+*anobjectinthedatabase.Emitawarningifthereferred-to+*objectdoesnotexist.+*/+staticintref_resolves_to_object(structref_entry*entry)+{+if(entry->flag&REF_ISBROKEN)+return0;+if(!has_sha1_file(entry->u.value.sha1)){+error("%s does not point to a valid object!",entry->name);+return0;+}+return1;+}++/**current_refisaperformancehack:wheniteratingoverreferences*usingthefor_each_ref*()functions,current_refissettothe*currentreference'sentrybeforecallingthecallbackfunction.If
@@ -549,14 +565,10 @@ static int do_one_ref(const char *base, each_ref_fn fn, int trim,if(prefixcmp(entry->name,base))return0;-if(!(flags&DO_FOR_EACH_INCLUDE_BROKEN)){-if(entry->flag&REF_ISBROKEN)-return0;/* ignore broken refs e.g. dangling symref */-if(!has_sha1_file(entry->u.value.sha1)){-error("%s does not point to a valid object!",entry->name);-return0;-}-}+if(!((flags&DO_FOR_EACH_INCLUDE_BROKEN)||+ref_resolves_to_object(entry)))+return0;+current_ref=entry;retval=fn(entry->name+trim,entry->u.value.sha1,entry->flag,cb_data);current_ref=NULL;
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
pack-refs.c doesn't contain much code, and the code it does contain is
closely related to reference handling. Moreover, there is some
duplication between pack_refs() and repack_without_ref(). Therefore,
merge pack-refs.c into refs.c and pack-refs.h into refs.h.
The code duplication will be addressed in future commits.
Signed-off-by: Michael Haggerty <redacted>
---
Makefile | 2 -
builtin/clone.c | 1 -
builtin/pack-refs.c | 2 +-
pack-refs.c | 148 ----------------------------------------------------
pack-refs.h | 18 -------
refs.c | 144 ++++++++++++++++++++++++++++++++++++++++++++++++++
refs.h | 14 +++++
7 files changed, 159 insertions(+), 170 deletions(-)
delete mode 100644 pack-refs.c
delete mode 100644 pack-refs.h
@@ -1,18 +0,0 @@-#ifndef PACK_REFS_H-#define PACK_REFS_H--/*- * Flags for controlling behaviour of pack_refs()- * PACK_REFS_PRUNE: Prune loose refs after packing- * PACK_REFS_ALL: Pack _all_ refs, not just tags and already packed refs- */-#define PACK_REFS_PRUNE 0x0001-#define PACK_REFS_ALL 0x0002--/*- * Write a packed-refs file for the current repository.- * flags: Combination of the above PACK_REFS_* flags.- */-int pack_refs(unsigned int flags);--#endif /* PACK_REFS_H */
@@ -1981,6 +1981,150 @@ static void write_packed_entry(int fd, char *refname, unsigned char *sha1,}}+structref_to_prune{+structref_to_prune*next;+unsignedcharsha1[20];+charname[FLEX_ARRAY];+};++structpack_refs_cb_data{+unsignedintflags;+structref_to_prune*ref_to_prune;+FILE*refs_file;+};++staticintdo_not_prune(intflags)+{+/* If it is already packed or if it is a symref,+*donotpruneit.+*/+return(flags&(REF_ISSYMREF|REF_ISPACKED));+}++staticintpack_one_ref(constchar*path,constunsignedchar*sha1,+intflags,void*cb_data)+{+structpack_refs_cb_data*cb=cb_data;+structobject*o;+intis_tag_ref;++/* Do not pack the symbolic refs */+if((flags&REF_ISSYMREF))+return0;+is_tag_ref=!prefixcmp(path,"refs/tags/");++/* ALWAYS pack refs that were already packed or are tags */+if(!(cb->flags&PACK_REFS_ALL)&&!is_tag_ref&&!(flags&REF_ISPACKED))+return0;++fprintf(cb->refs_file,"%s %s\n",sha1_to_hex(sha1),path);++o=parse_object_or_die(sha1,path);+if(o->type==OBJ_TAG){+o=deref_tag(o,path,0);+if(o)+fprintf(cb->refs_file,"^%s\n",+sha1_to_hex(o->sha1));+}++if((cb->flags&PACK_REFS_PRUNE)&&!do_not_prune(flags)){+intnamelen=strlen(path)+1;+structref_to_prune*n=xcalloc(1,sizeof(*n)+namelen);+hashcpy(n->sha1,sha1);+strcpy(n->name,path);+n->next=cb->ref_to_prune;+cb->ref_to_prune=n;+}+return0;+}++/*+*Removeemptyparents,butsparerefs/andimmediatesubdirs.+*Note:munges*name.+*/+staticvoidtry_remove_empty_parents(char*name)+{+char*p,*q;+inti;+p=name;+for(i=0;i<2;i++){/* refs/{heads,tags,...}/ */+while(*p&&*p!='/')+p++;+/* tolerate duplicate slashes; see check_refname_format() */+while(*p=='/')+p++;+}+for(q=p;*q;q++)+;+while(1){+while(q>p&&*q!='/')+q--;+while(q>p&&*(q-1)=='/')+q--;+if(q==p)+break;+*q='\0';+if(rmdir(git_path("%s",name)))+break;+}+}++/* make sure nobody touched the ref, and unlink */+staticvoidprune_ref(structref_to_prune*r)+{+structref_lock*lock=lock_ref_sha1(r->name+5,r->sha1);++if(lock){+unlink_or_warn(git_path("%s",r->name));+unlock_ref(lock);+try_remove_empty_parents(r->name);+}+}++staticvoidprune_refs(structref_to_prune*r)+{+while(r){+prune_ref(r);+r=r->next;+}+}++staticstructlock_filepacked;++intpack_refs(unsignedintflags)+{+intfd;+structpack_refs_cb_datacbdata;++memset(&cbdata,0,sizeof(cbdata));+cbdata.flags=flags;++fd=hold_lock_file_for_update(&packed,git_path("packed-refs"),+LOCK_DIE_ON_ERROR);+cbdata.refs_file=fdopen(fd,"w");+if(!cbdata.refs_file)+die_errno("unable to create ref-pack file structure");++/* perhaps other traits later as well */+fprintf(cbdata.refs_file,"# pack-refs with: peeled fully-peeled \n");++for_each_ref(pack_one_ref,&cbdata);+if(ferror(cbdata.refs_file))+die("failed to write ref-pack file");+if(fflush(cbdata.refs_file)||fsync(fd)||fclose(cbdata.refs_file))+die_errno("failed to write ref-pack file");+/*+*Sincethelockfilewasfdopen()'edandthenfclose()'edabove,+*assign-1tothelockfiledescriptorsothatcommit_lock_file()+*won'ttrytoclose()it.+*/+packed.fd=-1;+if(commit_lock_file(&packed)<0)+die_errno("unable to overwrite old ref-pack file");+prune_refs(cbdata.ref_to_prune);+return0;+}+staticintrepack_ref_fn(structref_entry*entry,void*cb_data){int*fd=cb_data;
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
A packed reference can be overridden by a loose reference, in which
case the packed reference is obsolete and is never used. The object
pointed to by such a reference can be garbage collected. Since
d66da478f2, this could lead to the emission of a spurious error
message:
error: refs/heads/master does not point to a valid object!
The error is generated by repack_without_ref() if there is an obsolete
dangling packed reference in packed-refs when the packed-refs file has
to be rewritten due to the deletion of another packed reference. Add
a failing test demonstrating this problem and some passing tests of
related scenarios.
Signed-off-by: Michael Haggerty <redacted>
---
How can I get rid of the sleeps in these tests? I couldn't find
another portable way of reliably ensuring that commits get
garbage-collected. (Some other tests use similar code without the
sleeps, but empirically the sleeps are necessary to get the tests to
work reliably. Perhaps the other tests are not doing what they
think.)
t/t3210-pack-refs.sh | 36 ++++++++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)
@@ -118,4 +118,40 @@ test_expect_success 'pack, prune and repack' 'test_cmpall-of-themagain'+test_expect_success'explicit pack-refs with dangling packed reference''+gitcommit--allow-empty-m"soon to be garbage-collected"&&+gitpack-refs--all&&+gitreset--hardHEAD^&&+sleep1&&+gitreflogexpire--expire=now--all&&+gitprune--expire=now&&+gitpack-refs--all2>result&&+test_cmp/dev/nullresult+'++test_expect_success'delete ref with dangling packed version''+gitcheckout-blamb&&+gitcommit--allow-empty-m"future garbage"&&+gitpack-refs--all&&+gitreset--hardHEAD^&&+gitcheckoutmaster&&+sleep1&&+gitreflogexpire--expire=now--all&&+gitprune--expire=now&&+gitbranch-dlamb2>result&&+test_cmp/dev/nullresult+'++test_expect_failure'delete ref while another dangling packed ref''+gitbranchlamb&&+gitcommit--allow-empty-m"future garbage"&&+gitpack-refs--all&&+gitreset--hardHEAD^&&+sleep1&&+gitreflogexpire--expire=now--all&&+gitprune--expire=now&&+gitbranch-dlamb2>result&&+test_cmp/dev/nullresult+'+ test_done
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:49
pack_refs() was not using any of the extra features of for_each_ref(),
so change it to use do_for_each_entry(). This also gives it access to
the ref_entry and in particular its peeled field, which will be taken
advantage of in the next commit.
Signed-off-by: Michael Haggerty <redacted>
---
refs.c | 29 +++++++++++++++--------------
1 file changed, 15 insertions(+), 14 deletions(-)
@@ -2001,37 +2001,38 @@ static int do_not_prune(int flags)return(flags&(REF_ISSYMREF|REF_ISPACKED));}-staticintpack_one_ref(constchar*refname,constunsignedchar*sha1,-intflags,void*cb_data)+staticintpack_one_ref(structref_entry*entry,void*cb_data){structpack_refs_cb_data*cb=cb_data;structobject*o;intis_tag_ref;-/* Do not pack the symbolic refs */-if((flags&REF_ISSYMREF))+/* Do not pack symbolic or broken refs: */+if((entry->flag&REF_ISSYMREF)||!ref_resolves_to_object(entry,0))return0;-is_tag_ref=!prefixcmp(refname,"refs/tags/");+is_tag_ref=!prefixcmp(entry->name,"refs/tags/");/* ALWAYS pack refs that were already packed or are tags */-if(!(cb->flags&PACK_REFS_ALL)&&!is_tag_ref&&!(flags&REF_ISPACKED))+if(!(cb->flags&PACK_REFS_ALL)&&!is_tag_ref&&+!(entry->flag&REF_ISPACKED))return0;-fprintf(cb->refs_file,"%s %s\n",sha1_to_hex(sha1),refname);+fprintf(cb->refs_file,"%s %s\n",sha1_to_hex(entry->u.value.sha1),+entry->name);-o=parse_object_or_die(sha1,refname);+o=parse_object_or_die(entry->u.value.sha1,entry->name);if(o->type==OBJ_TAG){-o=deref_tag(o,refname,0);+o=deref_tag(o,entry->name,0);if(o)fprintf(cb->refs_file,"^%s\n",sha1_to_hex(o->sha1));}-if((cb->flags&PACK_REFS_PRUNE)&&!do_not_prune(flags)){-intnamelen=strlen(refname)+1;+if((cb->flags&PACK_REFS_PRUNE)&&!do_not_prune(entry->flag)){+intnamelen=strlen(entry->name)+1;structref_to_prune*n=xcalloc(1,sizeof(*n)+namelen);-hashcpy(n->sha1,sha1);-strcpy(n->name,refname);+hashcpy(n->sha1,entry->u.value.sha1);+strcpy(n->name,entry->name);n->next=cb->ref_to_prune;cb->ref_to_prune=n;}
@@ -2108,7 +2109,7 @@ int pack_refs(unsigned int flags)/* perhaps other traits later as well */fprintf(cbdata.refs_file,"# pack-refs with: peeled fully-peeled \n");-for_each_ref(pack_one_ref,&cbdata);+do_for_each_entry(NULL,"",pack_one_ref,&cbdata);if(ferror(cbdata.refs_file))die("failed to write ref-pack file");if(fflush(cbdata.refs_file)||fsync(fd)||fclose(cbdata.refs_file))
@@ -2005,16 +2005,15 @@ static int pack_one_ref(struct ref_entry *entry, void *cb_data){structpack_refs_cb_data*cb=cb_data;enumpeel_statuspeel_status;-intis_tag_ref;+intis_tag_ref=!prefixcmp(entry->name,"refs/tags/");-/* Do not pack symbolic or broken refs: */-if((entry->flag&REF_ISSYMREF)||!ref_resolves_to_object(entry,0))+/* ALWAYS pack refs that were already packed or are tags */+if(!((cb->flags&PACK_REFS_ALL)||is_tag_ref||+(entry->flag&REF_ISPACKED)))return0;-is_tag_ref=!prefixcmp(entry->name,"refs/tags/");-/* ALWAYS pack refs that were already packed or are tags */-if(!(cb->flags&PACK_REFS_ALL)&&!is_tag_ref&&-!(entry->flag&REF_ISPACKED))+/* Do not pack symbolic or broken refs: */+if((entry->flag&REF_ISSYMREF)||!ref_resolves_to_object(entry,0))return0;peel_status=peel_entry(entry,1);
From: Junio C Hamano <hidden> Date: 2016-06-15 22:56:50
Michael Haggerty [off-list ref] writes:
Instead of just returning a success/failure bit, return an enumeration
value that explains the reason for any failure. This will come in
handy shortly.
Signed-off-by: Michael Haggerty <redacted>
---
This is a valid rewrite because all existing callers check if the
return value is 0 and does not check the error code against -1.
Looking good so far.
From: Junio C Hamano <hidden> Date: 2016-06-15 22:56:50
Michael Haggerty [off-list ref] writes:
The old version was inconsistent: when a reference was
REF_KNOWS_PEELED but with a null peeled value, it returned non-zero
for the current reference but zero for other references. Change the
behavior for non-current references to match that of current_ref,
which is what callers expect. Document the behavior.
Current callers did not trigger the previously-buggy behavior.
Is that because we were lucky by codeflow, or is it just that we
didn't have a testcase to trigger the behaviour?
From: Junio C Hamano <hidden> Date: 2016-06-15 22:56:50
Michael Haggerty [off-list ref] writes:
if (read_ref_full(refname, base, 1, &flag))
return -1;
- if ((flag & REF_ISPACKED)) {
+ /*
+ * If the reference is packed, read its ref_entry from the
+ * cache in the hope that we already know its peeled value.
+ * We only try this optimization on packed references because
+ * (a) forcing the filling of the loose reference cache could
+ * be expensive and (b) loose references anyway usually do not
+ * have REF_KNOWS_PEELED.
+ */
+ if (flag & REF_ISPACKED) {
struct ref_entry *r = get_packed_ref(refname);
This code makes the reader wonder what happens when a new loose ref
masks a stale packed ref, but the worry is unfounded because the
read_ref_full() wouldn't have gave us REF_ISPACKED in the flag in
such a case.
But somehow the calling sequence looks like such a mistake waiting
to happen. It would be much more clear if a function that returns a
"struct ref_entry *" is used instead of read_ref_full() above, and
we checked (r->flag & REF_ISPACKED) in the conditional, without a
separate get_packed_ref(refname).
-
- if (r && (r->flag & REF_KNOWS_PEELED)) {
- if (is_null_sha1(r->u.value.peeled))
- return -1;
+ if (r) {
+ if (peel_entry(r))
+ return -1;
hashcpy(sha1, r->u.value.peeled);
return 0;
}
From: Junio C Hamano <hidden> Date: 2016-06-15 22:56:50
Michael Haggerty [off-list ref] writes:
Establish an internal API for iterating over references, which gives
the callback functions direct access to the ref_entry structure
describing the reference. (Do not change the iteration API that is
exposed outside of the module.)
Define a new internal callback signature
int each_ref_entry_fn(struct ref_entry *entry, void *cb_data)
Change do_for_each_ref_in_dir() and do_for_each_ref_in_dirs() to
accept each_ref_entry_fn callbacks, and rename them to
do_for_each_entry_in_dir() and do_for_each_entry_in_dirs(),
respectively. Adapt their callers accordingly.
Add a new function do_for_each_entry() analogous to do_for_each_ref()
but using the new callback style.
Nicely done.
Change do_one_ref() into an each_ref_entry_fn that does some
bookkeeping and then calls a wrapped each_ref_fn.
Reimplement do_for_each_ref() in terms of do_for_each_entry(), using
do_one_ref() as an adapter.
Please note that the responsibility for setting current_ref remains in
do_one_ref(), which means that current_ref is *not* set when iterating
over references via the new internal API. This is not a disadvantage,
because current_ref is not needed by callers of the internal API (they
receive a pointer to the current ref_entry anyway). But more
importantly, this change prevents peel_ref() from returning invalid
results in the following scenario:
When iterating via the external API, the iteration always includes
both packed and loose references, and in particular never presents a
packed ref if there is a loose ref with the same name. The internal
API, on the other hand, gives the option to iterate over only the
packed references. During such an iteration, there is no check
whether the packed ref might be hidden by a loose ref of the same
name. But until now the packed ref was recorded in current_ref during
the iteration. So if peel_ref() were called with the reference name
corresponding to current ref, it would return the peeled version of
the packed ref even though there might be a loose ref that peels to a
different value. This scenario doesn't currently occur in the code,
but fix it to prevent things from breaking in a very confusing way in
the future.
Hopefully that means "in later patches in this series" ;-)
From: Junio C Hamano <hidden> Date: 2016-06-15 22:56:50
Michael Haggerty [off-list ref] writes:
A packed reference can be overridden by a loose reference, in which
case the packed reference is obsolete and is never used. The object
pointed to by such a reference can be garbage collected. Since
d66da478f2, this could lead to the emission of a spurious error
message:
error: refs/heads/master does not point to a valid object!
The error is generated by repack_without_ref() if there is an obsolete
dangling packed reference in packed-refs when the packed-refs file has
to be rewritten due to the deletion of another packed reference. Add
a failing test demonstrating this problem and some passing tests of
related scenarios.
That is one nasty recent bug.
Signed-off-by: Michael Haggerty <redacted>
---
How can I get rid of the sleeps in these tests?
From: Junio C Hamano <hidden> Date: 2016-06-15 22:56:50
Michael Haggerty [off-list ref] writes:
Stop emitting an error message for dangling packed references found
when deleting another packed reference. See the previous commit for a
longer explanation of the issue.
Change repack_without_ref_fn() to silently ignore dangling packed
references.
Signed-off-by: Michael Haggerty <redacted>
Somehow this feels as if it is sweeping the problem under the rug.
If you ignore a ref for which a loose ref exists when you update a
packed refs file, whether the stale "packed" one points at an object
that is still there or an object that has been garbage collected,
you would not even have to check if the "ref" resolves to object or
anything like that, no?
Am I missing something?
This one feels iffy in the otherwise pleasant-to-read series.
@@ -531,15 +531,17 @@ static void sort_ref_dir(struct ref_dir *dir)/**Returntrueiffthereferencedescribedbyentrycanberesolvedto-*anobjectinthedatabase.Emitawarningifthereferred-to-*objectdoesnotexist.+*anobjectinthedatabase.Ifreport_errorsistrue,emita+*warningifthereferred-toobjectdoesnotexist.*/-staticintref_resolves_to_object(structref_entry*entry)+staticintref_resolves_to_object(structref_entry*entry,intreport_errors){if(entry->flag&REF_ISBROKEN)return0;if(!has_sha1_file(entry->u.value.sha1)){-error("%s does not point to a valid object!",entry->name);+if(report_errors)+error("%s does not point to a valid object!",+entry->name);return0;}return1;
@@ -1897,8 +1899,9 @@ static int repack_without_ref_fn(struct ref_entry *entry, void *cb_data)if(!strcmp(data->refname,entry->name))return0;-if(!ref_resolves_to_object(entry))-return0;/* Skip broken refs */+/* Silently skip broken refs: */+if(!ref_resolves_to_object(entry,0))+return0;len=snprintf(line,sizeof(line),"%s %s\n",sha1_to_hex(entry->u.value.sha1),entry->name);/* this should not happen but just being defensive */
@@ -142,7 +142,7 @@ test_expect_success 'delete ref with dangling packed version' 'test_cmp/dev/nullresult'-test_expect_failure'delete ref while another dangling packed ref''+test_expect_success'delete ref while another dangling packed ref''gitbranchlamb&&gitcommit--allow-empty-m"future garbage"&&gitpack-refs--all&&
From: Junio C Hamano <hidden> Date: 2016-06-15 22:56:50
Michael Haggerty [off-list ref] writes:
When a reference that existed in the packed-refs file is deleted, the
packed-refs file must be rewritten. Previously, the file was
rewritten without any peeled refs, even if the file contained peeled
refs when it was read. This was not a bug, because the packed-refs
file header didn't claim that the file contained peeled values. But
it had a performance cost, because the repository would lose the
benefit of having precomputed peeled references until pack-refs was
run again.
Good.
Teach repack_without_ref() to write peeled refs to the packed-refs
file (regardless of whether they were present in the old version of
the file).
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:51
On 04/15/2013 07:38 PM, Junio C Hamano wrote:
quoted
+/*
+ * Call fn for each reference in the specified submodule for which the
+ * refname begins with base. If trim is non-zero, then trim that many
+ * characters off the beginning of each refname before passing the
+ * refname to fn. flags can be DO_FOR_EACH_INCLUDE_BROKEN to include
+ * broken references in the iteration.
+ */
Early termination due to "fn()" returning non-zero needs to be
documented here, no?
quoted
static int do_for_each_ref(const char *submodule, const char *base, each_ref_fn fn,
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:51
On 04/15/2013 07:38 PM, Junio C Hamano wrote:
Michael Haggerty [off-list ref] writes:
quoted
The old version was inconsistent: when a reference was
REF_KNOWS_PEELED but with a null peeled value, it returned non-zero
for the current reference but zero for other references. Change the
behavior for non-current references to match that of current_ref,
which is what callers expect. Document the behavior.
Current callers did not trigger the previously-buggy behavior.
Is that because we were lucky by codeflow, or is it just that we
didn't have a testcase to trigger the behaviour?
Existing callers only called peel_ref() from within a for_each_ref-style
iteration and only for the current ref. Therefore the buggy code path
was impossible to reach.
I will note that in the commit message.
Michael
--
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:51
On 04/15/2013 07:38 PM, Junio C Hamano wrote:
Michael Haggerty [off-list ref] writes:
quoted
if (read_ref_full(refname, base, 1, &flag))
return -1;
- if ((flag & REF_ISPACKED)) {
+ /*
+ * If the reference is packed, read its ref_entry from the
+ * cache in the hope that we already know its peeled value.
+ * We only try this optimization on packed references because
+ * (a) forcing the filling of the loose reference cache could
+ * be expensive and (b) loose references anyway usually do not
+ * have REF_KNOWS_PEELED.
+ */
+ if (flag & REF_ISPACKED) {
struct ref_entry *r = get_packed_ref(refname);
This code makes the reader wonder what happens when a new loose ref
masks a stale packed ref, but the worry is unfounded because the
read_ref_full() wouldn't have gave us REF_ISPACKED in the flag in
such a case.
But somehow the calling sequence looks like such a mistake waiting
to happen. It would be much more clear if a function that returns a
"struct ref_entry *" is used instead of read_ref_full() above, and
we checked (r->flag & REF_ISPACKED) in the conditional, without a
separate get_packed_ref(refname).
As I'm sure you realize, I didn't change the code that you are referring
to; I just added a comment.
But yes, I sympathize with your complaint. Additionally, the code has
the drawback that get_packed_ref() is called twice: once in
read_ref_full() and again in the if block here. Unfortunately, this
isn't so easy to fix because read_ref_full() doesn't use the loose
reference cache, so the reference that it returns might not even have a
ref_entry associated with it (specifically, unless the returned flag
value has REF_ISPACKED set). So there are a couple options:
* Always read loose references through the cache; that way there would
always be a ref_entry in which the return value could be presented.
This would not be a good idea at the moment because the loose reference
cache is populated one directory at a time, and reading a whole
directory of loose references could be expensive. So before
implementing this, it would be advisable to change the code to populate
the loose reference cache more selectively when single loose references
are needed. -> This approach would be well beyond the scope of this
patch series.
* Implement a function like read_ref_full() with an additional (struct
ref_entry **entry) argument that is written to *in the case* that the
reference that was returned has a ref_entry associated with it, and NULL
otherwise. This would have to be an internal function because we don't
want to expose the ref_entry structure outside of refs.c.
read_ref_full() would be implemented on top of the new function.
Either way, I'd rather put this idea on my TODO list for another time.
Michael
--
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:51
On 04/15/2013 07:38 PM, Junio C Hamano wrote:
Michael Haggerty [off-list ref] writes:
quoted
[...] But more
importantly, this change prevents peel_ref() from returning invalid
results in the following scenario:
When iterating via the external API, the iteration always includes
both packed and loose references, and in particular never presents a
packed ref if there is a loose ref with the same name. The internal
API, on the other hand, gives the option to iterate over only the
packed references. During such an iteration, there is no check
whether the packed ref might be hidden by a loose ref of the same
name. But until now the packed ref was recorded in current_ref during
the iteration. So if peel_ref() were called with the reference name
corresponding to current ref, it would return the peeled version of
the packed ref even though there might be a loose ref that peels to a
different value. This scenario doesn't currently occur in the code,
but fix it to prevent things from breaking in a very confusing way in
the future.
Hopefully that means "in later patches in this series" ;-)
I don't think that the rest of the series would have triggered this
problem either. In fact, if I had written repack_without_ref()'s
peeling functionality using peel_ref(), then it would have *depended* on
this bug for its proper operation...otherwise it would have written the
peeled version of the loose ref to the packed-ref file. Of course, it's
all pretty academic because the peeled version of a packed ref should
never be used when it is overridden by a loose ref, so the incorrect
peeled values in the packed-ref file shouldn't have any observable effects.
The real problem is that calling the old peel_ref() function on a packed
reference was illegitimate because the function only knew how to peel a
ref that was still active. Plus it's kindof silly tucking away the
current reference in a global variable then looking it up again instead
of passing the ref_entry around.
Callers outside of refs.c could also not have triggered this bug because
they have no way to access overridden packed refs.
Michael
--
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:51
On 04/15/2013 07:39 PM, Junio C Hamano wrote:
Michael Haggerty [off-list ref] writes:
quoted
How can I get rid of the sleeps in these tests?
Would test-chmtime help?
Maybe I should take a step back and ask why it isn't easier to expire
things regardless of age, which is sometimes a reasonable thing to do
even outside of test suites. In particular, it seem to me that the most
obvious interpretation of
git reflog expire --expire=now --all
would be that it expires *everything*. But in fact it seems to only
expire things that are at least one second old, which doesn't seem at
all useful in the real world. "--expire=all" is accepted without
complaint but doesn't do what one would hope. Something like
"--expire=$(($(date +%s)+3600))" works, but it is not very convenient
(is it portable?).
I guess I can use test-chmtime for my particular test, though I will
have to pass it the explicit names of the logfile(s), like
find .git/logs -type f -print0 | xargs -0 test-chmtime =-60
I guess that's what I'll do if no better solution comes up.
Michael
--
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/
From: Michael Haggerty <hidden> Date: 2016-06-15 22:56:52
On 04/15/2013 07:39 PM, Junio C Hamano wrote:
Michael Haggerty [off-list ref] writes:
quoted
Stop emitting an error message for dangling packed references found
when deleting another packed reference. See the previous commit for a
longer explanation of the issue.
Change repack_without_ref_fn() to silently ignore dangling packed
references.
Signed-off-by: Michael Haggerty <redacted>
Somehow this feels as if it is sweeping the problem under the rug.
If you ignore a ref for which a loose ref exists when you update a
packed refs file, whether the stale "packed" one points at an object
that is still there or an object that has been garbage collected,
you would not even have to check if the "ref" resolves to object or
anything like that, no?
Am I missing something?
This one feels iffy in the otherwise pleasant-to-read series.
The usual situation when this code would be triggered would be that the
packed reference is overridden by a loose ref and points at an object
that has been garbage collected. In that case it is definitely
incorrect to emit an error message.
But the fact that we don't explicitly verify that there is an overriding
loose reference means that it is possible that the failure to resolve
the packed ref comes from some kind of repository corruption, and you
are correct that such a problem would be swept under the rug by my change.
I've been trying to minimize the extra work that repack_without_ref()
needs to do to write peeled references, to avoid stretching out the
delay that can now occur when deleting a reference. Thus I was trying
to save a check of loose references during this operation. But I guess
I agree that a little bit more caution would be prudent.
I can think of a few ways to avoid sweeping possible indications of repo
corruption under the rug, in order of increasing run-time:
1. If a packed ref's SHA-1 cannot be resolved, write the packed ref to
the new packed-refs file anyway with SHA-1 but without a peeled value.
This would avoid having to check the loose references and avoid erasing
possible evidence of corruption, but would delay an actual check for
corruption until a later time. It would be a quick fix, effectively
kicking the can down the road instead of sweeping it under the rug.
Minor pitfall: a reference that is listed without a peeled value in a
fully-peeled pack-refs file tells future readers that the corresponding
SHA-1 *cannot* be peeled. IF the named object would somehow reappear in
the repository (e.g., via a fetch) and IF the object is peelable and IF
there is in fact no loose ref overriding the packed ref, then the final
result would be that one form of corruption (reference points to
non-existent object) would be converted to another form (reference
falsely believed to be non-peelable). I think this is an acceptable
risk because (a) it would only happen in an unlikely series of events in
a repo that was already corrupt, and (b) because falsely believing a
reference to be non-peelable wouldn't have terrible consequences.
2. Whenever a packed reference cannot be resolved to an object, verify
that there is indeed a loose reference overriding it; if not, emit an
error and in either case omit the packed ref from the output.
3. Check for an overriding loose reference *before* trying to peel a
packed reference, and omit any overridden loose references from the
output packed-refs file. This would be close to running "pack-refs
--no-prune" without the "is_tag_ref" test and with reuse of available
peeled values. This approach would tidy up the packed-refs file a bit
more than (2) because it would cause the deletion of more overridden
packed refs, but only as part of first peeling them, which should only
happen once in a repo, and only if the first peeling occurs within
repack_without_ref() as opposed to an explicit pack_refs(). So it's a
negligible improvement over (2).
4. Further along the "correctness" spectrum, one could check for
overriding loose references *every* time the packed-refs file is
rewritten by repack_without_ref(), even for references whose peeled
values are already known. But this would add overhead to every deletion
of a packed reference, which is probably not justified.
I'm worried that implementing 2-4 would introduce new race conditions of
the type that Peff discovered recently, unless we fix the locking policy
first (which is also on my TODO list). So my suggestion is to implement
1 now and implement 2 sometime in the future.
Opinions?
Michael
--
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/