From: Jeff King <hidden> Date: 2016-06-15 23:07:31
I noticed that an interrupt "git symbolic-ref" will not clean up
"HEAD.lock". So I started this series as an attempt to convert
create_symref() to "struct lock_file" to get the usual tempfile cleanup.
But I found a few other points of interest. The biggest is that
git-symbolic-ref does not actually propagate errors in its exit code.
Whoops. That's fixed in the first patch, which could go separately to
"maint".
The rest of it is fairly dependent on master because of the
refs/files-backend.c code movement. I can backport it if we really want
it for maint.
[1/4]: symbolic-ref: propagate error code from create_symref()
[2/4]: t1401: test reflog creation for git-symbolic-ref
[3/4]: create_symref: modernize variable names
[4/4]: create_symref: use existing ref-lock code
-Peff
From: Jeff King <hidden> Date: 2016-06-15 23:07:31
If create_symref() fails, git-symbolic-ref will still exit
with code 0, and our caller has no idea that the command did
nothing.
This appears to have been broken since the beginning of time
(e.g., it is not a regression where create_symref() stopped
calling die() or something similar).
Signed-off-by: Jeff King <redacted>
---
builtin/symbolic-ref.c | 2 +-
t/t1401-symbolic-ref.sh | 6 ++++++
2 files changed, 7 insertions(+), 1 deletion(-)
@@ -67,7 +67,7 @@ int cmd_symbolic_ref(int argc, const char **argv, const char *prefix)if(!strcmp(argv[0],"HEAD")&&!starts_with(argv[1],"refs/"))die("Refusing to point HEAD outside of refs/");-create_symref(argv[0],argv[1],msg);+ret=!!create_symref(argv[0],argv[1],msg);break;default:usage_with_options(git_symbolic_ref_usage,options);
From: Jeff King <hidden> Date: 2016-06-15 23:07:31
The current code writes a reflog entry whenever we update a
symbolic ref, but we never test that this is so. Let's add a
test to make sure upcoming refactoring doesn't cause a
regression.
Signed-off-by: Jeff King <redacted>
---
t/t1401-symbolic-ref.sh | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
From: Jeff King <hidden> Date: 2016-06-15 23:07:31
Once upon a time, create_symref() was used only to point
HEAD at a branch name, and the variable names reflect that
(e.g., calling the path git_HEAD). However, it is much more
generic these days (and has been for some time). Let's
update the variable names to make it easier to follow:
- `ref_target` is now just `ref`, matching the declaration
in `cache.h` (and hopefully making it clear that it is
the symref itself, and not the target).
- `git_HEAD` is now `ref_path`; the on-disk path
corresponding to `ref`.
- `refs_heads_master` is now just `target`; i.e., what the
symref points at. This term also matches what is in
the symlink(2) manpage (at least on Linux).
- the buffer to hold the symref file's contents was simply
called `ref`. It's now `buf` (admittedly also generic,
but at least not actively introducing confusion with the
other variable holding the refname).
Signed-off-by: Jeff King <redacted>
---
This could actually be squashed in with patch 4, as most of the changed
instances just go away. I did find the new names easier to work with,
though, so maybe they make the diff for that patch easier.
refs.h | 2 +-
refs/files-backend.c | 41 ++++++++++++++++++++---------------------
2 files changed, 21 insertions(+), 22 deletions(-)
@@ -2811,58 +2811,57 @@ static int commit_ref_update(struct ref_lock *lock,return0;}-intcreate_symref(constchar*ref_target,constchar*refs_heads_master,-constchar*logmsg)+intcreate_symref(constchar*ref,constchar*target,constchar*logmsg){char*lockpath=NULL;-charref[1000];+charbuf[1000];intfd,len,written;-char*git_HEAD=git_pathdup("%s",ref_target);+char*ref_path=git_pathdup("%s",ref);unsignedcharold_sha1[20],new_sha1[20];structstrbuferr=STRBUF_INIT;-if(logmsg&&read_ref(ref_target,old_sha1))+if(logmsg&&read_ref(ref,old_sha1))hashclr(old_sha1);-if(safe_create_leading_directories(git_HEAD)<0)-returnerror("unable to create directory for %s",git_HEAD);+if(safe_create_leading_directories(ref_path)<0)+returnerror("unable to create directory for %s",ref_path);#ifndef NO_SYMLINK_HEADif(prefer_symlink_refs){-unlink(git_HEAD);-if(!symlink(refs_heads_master,git_HEAD))+unlink(ref_path);+if(!symlink(target,ref_path))gotodone;fprintf(stderr,"no symlink - falling back to symbolic ref\n");}#endif-len=snprintf(ref,sizeof(ref),"ref: %s\n",refs_heads_master);-if(sizeof(ref)<=len){-error("refname too long: %s",refs_heads_master);+len=snprintf(buf,sizeof(buf),"ref: %s\n",target);+if(sizeof(buf)<=len){+error("refname too long: %s",target);gotoerror_free_return;}-lockpath=mkpathdup("%s.lock",git_HEAD);+lockpath=mkpathdup("%s.lock",ref_path);fd=open(lockpath,O_CREAT|O_EXCL|O_WRONLY,0666);if(fd<0){error("Unable to open %s for writing",lockpath);gotoerror_free_return;}-written=write_in_full(fd,ref,len);+written=write_in_full(fd,buf,len);if(close(fd)!=0||written!=len){error("Unable to write to %s",lockpath);gotoerror_unlink_return;}-if(rename(lockpath,git_HEAD)<0){-error("Unable to create %s",git_HEAD);+if(rename(lockpath,ref_path)<0){+error("Unable to create %s",ref_path);gotoerror_unlink_return;}-if(adjust_shared_perm(git_HEAD)){+if(adjust_shared_perm(ref_path)){error("Unable to fix permissions on %s",lockpath);error_unlink_return:unlink_or_warn(lockpath);error_free_return:free(lockpath);-free(git_HEAD);+free(ref_path);return-1;}free(lockpath);
From: Michael Haggerty <hidden> Date: 2016-06-15 23:07:34
On 12/20/2015 08:29 AM, Jeff King wrote:
Once upon a time, create_symref() was used only to point
HEAD at a branch name, and the variable names reflect that
(e.g., calling the path git_HEAD). However, it is much more
generic these days (and has been for some time). Let's
update the variable names to make it easier to follow:
- `ref_target` is now just `ref`, matching the declaration
in `cache.h` (and hopefully making it clear that it is
the symref itself, and not the target).
I've been trying to name variables that hold reference names "refname"
to distinguish them clearly from other representations of references,
like "struct ref_entry *".
[...]
Otherwise LGTM.
Michael
--
Michael Haggerty
mhagger@alum.mit.edu
From: Jeff King <hidden> Date: 2016-06-15 23:07:34
On Mon, Dec 28, 2015 at 09:20:42AM +0100, Michael Haggerty wrote:
On 12/20/2015 08:29 AM, Jeff King wrote:
quoted
Once upon a time, create_symref() was used only to point
HEAD at a branch name, and the variable names reflect that
(e.g., calling the path git_HEAD). However, it is much more
generic these days (and has been for some time). Let's
update the variable names to make it easier to follow:
- `ref_target` is now just `ref`, matching the declaration
in `cache.h` (and hopefully making it clear that it is
the symref itself, and not the target).
I've been trying to name variables that hold reference names "refname"
to distinguish them clearly from other representations of references,
like "struct ref_entry *".
That makes sense. I've tweaked this patch to use "refname", and carried
through the change to the following patch.
-Peff
From: Jeff King <hidden> Date: 2016-06-15 23:07:31
The create_symref() function predates the existence of
"struct lock_file", let alone the more recent "struct
ref_lock". Instead, it just does its own manual dot-locking.
Besides being more code, this has a few downsides:
- if git is interrupted while holding the lock, we don't
clean up the lockfile
- we don't do the usual directory/filename conflict check.
So you can sometimes create a symref "refs/heads/foo/bar",
even if "refs/heads/foo" exists (namely, if the refs are
packed and we do not hit the d/f conflict in the
filesystem).
This patch refactors create_symref() to use the "struct
ref_lock" interface, which handles both of these things.
There are a few bonus cleanups that come along with it:
- we leaked ref_path in some error cases
- the symref contents were stored in a fixed-size buffer,
putting an artificial (albeit large) limitation on the
length of the refname. We now write through fprintf, and
handle refnames of any size.
- we called adjust_shared_perm only after the file was
renamed into place, creating a potential race with
readers in a shared repository. Now we fix the
permissions first, and commit only if that succeeded.
This also makes the update atomic with respect to our
exit code (whereas previously, we might report failure
even though we updated the ref).
- the legacy prefer_symlink_refs path did not do any
locking at all. Admittedly, it is not atomic from a
reader's perspective (and it cannot be; it has to unlink
and then symlink, creating a race), but at least it
cannot conflict with other writers now.
- the result of this patch is hopefully more readable. It
eliminates three goto labels. Two were for error checking
that is now simplified, and the third was to reach shared
code that has been pulled into its own function.
Signed-off-by: Jeff King <redacted>
---
refs/files-backend.c | 113 +++++++++++++++++++++++++-----------------------
t/t1401-symbolic-ref.sh | 8 ++++
2 files changed, 66 insertions(+), 55 deletions(-)
@@ -2811,74 +2811,77 @@ static int commit_ref_update(struct ref_lock *lock,return0;}-intcreate_symref(constchar*ref,constchar*target,constchar*logmsg)+staticintcreate_ref_symlink(structref_lock*lock,constchar*target){-char*lockpath=NULL;-charbuf[1000];-intfd,len,written;-char*ref_path=git_pathdup("%s",ref);-unsignedcharold_sha1[20],new_sha1[20];-structstrbuferr=STRBUF_INIT;--if(logmsg&&read_ref(ref,old_sha1))-hashclr(old_sha1);--if(safe_create_leading_directories(ref_path)<0)-returnerror("unable to create directory for %s",ref_path);-+intret=-1;#ifndef NO_SYMLINK_HEAD-if(prefer_symlink_refs){-unlink(ref_path);-if(!symlink(target,ref_path))-gotodone;+char*ref_path=get_locked_file_path(lock->lk);+unlink(ref_path);+ret=symlink(target,ref_path);+free(ref_path);++if(ret)fprintf(stderr,"no symlink - falling back to symbolic ref\n");-}#endif+returnret;+}-len=snprintf(buf,sizeof(buf),"ref: %s\n",target);-if(sizeof(buf)<=len){-error("refname too long: %s",target);-gotoerror_free_return;-}-lockpath=mkpathdup("%s.lock",ref_path);-fd=open(lockpath,O_CREAT|O_EXCL|O_WRONLY,0666);-if(fd<0){-error("Unable to open %s for writing",lockpath);-gotoerror_free_return;-}-written=write_in_full(fd,buf,len);-if(close(fd)!=0||written!=len){-error("Unable to write to %s",lockpath);-gotoerror_unlink_return;-}-if(rename(lockpath,ref_path)<0){-error("Unable to create %s",ref_path);-gotoerror_unlink_return;-}-if(adjust_shared_perm(ref_path)){-error("Unable to fix permissions on %s",lockpath);-error_unlink_return:-unlink_or_warn(lockpath);-error_free_return:-free(lockpath);-free(ref_path);-return-1;-}-free(lockpath);--#ifndef NO_SYMLINK_HEAD-done:-#endif+staticvoidupdate_symref_reflog(structref_lock*lock,constchar*ref,+constchar*target,constchar*logmsg)+{+structstrbuferr=STRBUF_INIT;+unsignedcharnew_sha1[20];if(logmsg&&!read_ref(target,new_sha1)&&-log_ref_write(ref,old_sha1,new_sha1,logmsg,0,&err)){+log_ref_write(ref,lock->old_oid.hash,new_sha1,logmsg,0,&err)){error("%s",err.buf);strbuf_release(&err);}+}-free(ref_path);+staticintcreate_symref_locked(structref_lock*lock,constchar*ref,+constchar*target,constchar*logmsg)+{+if(prefer_symlink_refs&&!create_ref_symlink(lock,target)){+update_symref_reflog(lock,ref,target,logmsg);+return0;+}++if(!fdopen_lock_file(lock->lk,"w"))+returnerror("unable to fdopen %s: %s",+lock->lk->tempfile.filename.buf,strerror(errno));++if(adjust_shared_perm(lock->lk->tempfile.filename.buf))+returnerror("unable to fix permissions on %s: %s",+lock->lk->tempfile.filename.buf,strerror(errno));++/* no error check; commit_ref will check ferror */+fprintf(lock->lk->tempfile.fp,"ref: %s\n",target);+if(commit_ref(lock)<0)+returnerror("unable to write symref for %s: %s",ref,+strerror(errno));+update_symref_reflog(lock,ref,target,logmsg);return0;}+intcreate_symref(constchar*ref,constchar*target,constchar*logmsg)+{+structstrbuferr=STRBUF_INIT;+structref_lock*lock;+intret;++lock=lock_ref_sha1_basic(ref,NULL,NULL,NULL,REF_NODEREF,NULL,+&err);+if(!lock){+error("%s",err.buf);+strbuf_release(&err);+return-1;+}++ret=create_symref_locked(lock,ref,target,logmsg);+unlock_ref(lock);+returnret;+}+intreflog_exists(constchar*refname){structstatst;
From: Michael Haggerty <hidden> Date: 2016-06-15 23:07:34
On 12/20/2015 08:34 AM, Jeff King wrote:
quoted hunk
The create_symref() function predates the existence of
"struct lock_file", let alone the more recent "struct
ref_lock". Instead, it just does its own manual dot-locking.
Besides being more code, this has a few downsides:
- if git is interrupted while holding the lock, we don't
clean up the lockfile
- we don't do the usual directory/filename conflict check.
So you can sometimes create a symref "refs/heads/foo/bar",
even if "refs/heads/foo" exists (namely, if the refs are
packed and we do not hit the d/f conflict in the
filesystem).
This patch refactors create_symref() to use the "struct
ref_lock" interface, which handles both of these things.
There are a few bonus cleanups that come along with it:
- we leaked ref_path in some error cases
- the symref contents were stored in a fixed-size buffer,
putting an artificial (albeit large) limitation on the
length of the refname. We now write through fprintf, and
handle refnames of any size.
- we called adjust_shared_perm only after the file was
renamed into place, creating a potential race with
readers in a shared repository. Now we fix the
permissions first, and commit only if that succeeded.
This also makes the update atomic with respect to our
exit code (whereas previously, we might report failure
even though we updated the ref).
- the legacy prefer_symlink_refs path did not do any
locking at all. Admittedly, it is not atomic from a
reader's perspective (and it cannot be; it has to unlink
and then symlink, creating a race), but at least it
cannot conflict with other writers now.
- the result of this patch is hopefully more readable. It
eliminates three goto labels. Two were for error checking
that is now simplified, and the third was to reach shared
code that has been pulled into its own function.
Signed-off-by: Jeff King <redacted>
---
refs/files-backend.c | 113 +++++++++++++++++++++++++-----------------------
t/t1401-symbolic-ref.sh | 8 ++++
2 files changed, 66 insertions(+), 55 deletions(-)
@@ -2811,74 +2811,77 @@ static int commit_ref_update(struct ref_lock *lock,return0;}-intcreate_symref(constchar*ref,constchar*target,constchar*logmsg)+staticintcreate_ref_symlink(structref_lock*lock,constchar*target){-char*lockpath=NULL;-charbuf[1000];-intfd,len,written;-char*ref_path=git_pathdup("%s",ref);-unsignedcharold_sha1[20],new_sha1[20];-structstrbuferr=STRBUF_INIT;--if(logmsg&&read_ref(ref,old_sha1))-hashclr(old_sha1);--if(safe_create_leading_directories(ref_path)<0)-returnerror("unable to create directory for %s",ref_path);-+intret=-1;#ifndef NO_SYMLINK_HEAD-if(prefer_symlink_refs){-unlink(ref_path);-if(!symlink(target,ref_path))-gotodone;+char*ref_path=get_locked_file_path(lock->lk);+unlink(ref_path);+ret=symlink(target,ref_path);+free(ref_path);++if(ret)fprintf(stderr,"no symlink - falling back to symbolic ref\n");-}#endif+returnret;+}
This function is racy. A reader might see no reference at all in the
moment between the `unlink()` and the `symlink()`. Moreover, if this
process is killed at that moment, the symbolic ref would be gone forever.
I think that the semantics of `rename()` would allow this race to be
fixed, though, since `symlink()` doesn't have the analogue of
`O_CREAT|O_EXCL`, one would need a lockfile *and* a second temporary
filename under which the new symlink is originally created.
However, this race has always been here, and symlink-based symrefs are
obsolete, so it's probably not worth fixing.
You can skip this step. lock_file() already calls adjust_shared_perm().
+ /* no error check; commit_ref will check ferror */
+ fprintf(lock->lk->tempfile.fp, "ref: %s\n", target);
+ if (commit_ref(lock) < 0)
+ return error("unable to write symref for %s: %s", ref,
+ strerror(errno));
+ update_symref_reflog(lock, ref, target, logmsg);
Here is another problem that didn't originate with your changes:
The reflog should be written while holding the reference lock, to
prevent two processes' trying to write new entries at the same time.
I think the problem would be solved if you move the call to
update_symref_reflog() above the call to commit_ref().
Granted, this could case a reflog entry to be written for a reference
update whose commit fails, but that's also a risk for non-symbolic
references. Fixing this residual problem would require the ability to
roll back reflog changes.
[...]
This function is racy. A reader might see no reference at all in the
moment between the `unlink()` and the `symlink()`. Moreover, if this
process is killed at that moment, the symbolic ref would be gone forever.
I think that the semantics of `rename()` would allow this race to be
fixed, though, since `symlink()` doesn't have the analogue of
`O_CREAT|O_EXCL`, one would need a lockfile *and* a second temporary
filename under which the new symlink is originally created.
However, this race has always been here, and symlink-based symrefs are
obsolete, so it's probably not worth fixing.
Yeah. In the commit message I wrote:
quoted
- the legacy prefer_symlink_refs path did not do any
locking at all. Admittedly, it is not atomic from a
reader's perspective (and it cannot be; it has to unlink
and then symlink, creating a race), but at least it
cannot conflict with other writers now.
but I think you are right that we could do tricks with rename() on the
symlink. That is in POSIX, though I wouldn't be surprised if it
misbehaves on some obscure systems. Given that using symlinks is only
triggered by an undocumented (!) option that presumably very few people
use, I'm inclined to leave it as-is.
I was actually tempted to rip it out, as the option is mostly a hack
from 2006:
http://article.gmane.org/gmane.comp.version-control.git/19402
It was done to let people bisect old kernel trees whose build system
_depends_ on .git/HEAD being a symlink. My thought is:
- people are a lot less likely to bisect back that far anymore
- ...and if they do, their build system will be totally broken by the
new ref-backend stuff
- ...and the right solution is not to put a hack in git, but for the
bisecting user to "fix up" the tree before testing (by munging the
symlink themselves, or applying a patch to the build system to use
`git rev-parse`). This matches what people have to do for every
other type of weird "the old version doesn't quite build"
incompatibility that has nothing to do with git.
But that should probably come as a separate patch (it affects this only
in that it makes this patch simpler to do that cleanup first :) ).
quoted
+ if (adjust_shared_perm(lock->lk->tempfile.filename.buf))
+ return error("unable to fix permissions on %s: %s",
+ lock->lk->tempfile.filename.buf, strerror(errno));
You can skip this step. lock_file() already calls adjust_shared_perm().
Thanks, fixed.
quoted
+ /* no error check; commit_ref will check ferror */
+ fprintf(lock->lk->tempfile.fp, "ref: %s\n", target);
+ if (commit_ref(lock) < 0)
+ return error("unable to write symref for %s: %s", ref,
+ strerror(errno));
+ update_symref_reflog(lock, ref, target, logmsg);
Here is another problem that didn't originate with your changes:
The reflog should be written while holding the reference lock, to
prevent two processes' trying to write new entries at the same time.
I think the problem would be solved if you move the call to
update_symref_reflog() above the call to commit_ref().
Granted, this could case a reflog entry to be written for a reference
update whose commit fails, but that's also a risk for non-symbolic
references. Fixing this residual problem would require the ability to
roll back reflog changes.
Thanks, I agree with your reasoning (and especially that we should err
on the same side that normal ref-writing does). I'll do it in a
follow-on patch, though, to make it more clear what is going on.
-Peff
From: Jeff King <hidden> Date: 2016-06-15 23:07:34
On Sun, Dec 20, 2015 at 02:26:37AM -0500, Jeff King wrote:
I noticed that an interrupt "git symbolic-ref" will not clean up
"HEAD.lock". So I started this series as an attempt to convert
create_symref() to "struct lock_file" to get the usual tempfile cleanup.
Here's version 2, based on comments from Michael. The first two patches
were picked out separately for jk/symbolic-ref-maint, so I've dropped
them here (so 1+2 here are the original 3+4).
The other differences from v1 are:
- use "refname" instead of "ref" to match surrounding code
- drop adjust_shared_perm, as lockfile does it for us
- adjust reflog writing order (done in a new patch)
The patches are:
[1/3]: create_symref: modernize variable names
[2/3]: create_symref: use existing ref-lock code
[3/3]: create_symref: write reflog while holding lock
-Peff
From: Jeff King <hidden> Date: 2016-06-15 23:07:34
Once upon a time, create_symref() was used only to point
HEAD at a branch name, and the variable names reflect that
(e.g., calling the path git_HEAD). However, it is much more
generic these days (and has been for some time). Let's
update the variable names to make it easier to follow:
- `ref_target` is now just `refname`. This is closer to
the `ref` that is already in `cache.h`, but with the
extra twist that "name" makes it clear this is the name
and not a ref struct. Dropping "target" hopefully makes
it clear that we are talking about the symref itself,
not what it points to.
- `git_HEAD` is now `ref_path`; the on-disk path
corresponding to `ref`.
- `refs_heads_master` is now just `target`; i.e., what the
symref points at. This term also matches what is in
the symlink(2) manpage (at least on Linux).
- the buffer to hold the symref file's contents was simply
called `ref`. It's now `buf` (admittedly also generic,
but at least not actively introducing confusion with the
other variable holding the refname).
Signed-off-by: Jeff King <redacted>
---
refs.h | 2 +-
refs/files-backend.c | 41 ++++++++++++++++++++---------------------
2 files changed, 21 insertions(+), 22 deletions(-)
@@ -2811,58 +2811,57 @@ static int commit_ref_update(struct ref_lock *lock,return0;}-intcreate_symref(constchar*ref_target,constchar*refs_heads_master,-constchar*logmsg)+intcreate_symref(constchar*refname,constchar*target,constchar*logmsg){char*lockpath=NULL;-charref[1000];+charbuf[1000];intfd,len,written;-char*git_HEAD=git_pathdup("%s",ref_target);+char*ref_path=git_pathdup("%s",refname);unsignedcharold_sha1[20],new_sha1[20];structstrbuferr=STRBUF_INIT;-if(logmsg&&read_ref(ref_target,old_sha1))+if(logmsg&&read_ref(refname,old_sha1))hashclr(old_sha1);-if(safe_create_leading_directories(git_HEAD)<0)-returnerror("unable to create directory for %s",git_HEAD);+if(safe_create_leading_directories(ref_path)<0)+returnerror("unable to create directory for %s",ref_path);#ifndef NO_SYMLINK_HEADif(prefer_symlink_refs){-unlink(git_HEAD);-if(!symlink(refs_heads_master,git_HEAD))+unlink(ref_path);+if(!symlink(target,ref_path))gotodone;fprintf(stderr,"no symlink - falling back to symbolic ref\n");}#endif-len=snprintf(ref,sizeof(ref),"ref: %s\n",refs_heads_master);-if(sizeof(ref)<=len){-error("refname too long: %s",refs_heads_master);+len=snprintf(buf,sizeof(buf),"ref: %s\n",target);+if(sizeof(buf)<=len){+error("refname too long: %s",target);gotoerror_free_return;}-lockpath=mkpathdup("%s.lock",git_HEAD);+lockpath=mkpathdup("%s.lock",ref_path);fd=open(lockpath,O_CREAT|O_EXCL|O_WRONLY,0666);if(fd<0){error("Unable to open %s for writing",lockpath);gotoerror_free_return;}-written=write_in_full(fd,ref,len);+written=write_in_full(fd,buf,len);if(close(fd)!=0||written!=len){error("Unable to write to %s",lockpath);gotoerror_unlink_return;}-if(rename(lockpath,git_HEAD)<0){-error("Unable to create %s",git_HEAD);+if(rename(lockpath,ref_path)<0){+error("Unable to create %s",ref_path);gotoerror_unlink_return;}-if(adjust_shared_perm(git_HEAD)){+if(adjust_shared_perm(ref_path)){error("Unable to fix permissions on %s",lockpath);error_unlink_return:unlink_or_warn(lockpath);error_free_return:free(lockpath);-free(git_HEAD);+free(ref_path);return-1;}free(lockpath);
From: Jeff King <hidden> Date: 2016-06-15 23:07:34
We generally hold a lock on the matching ref while writing
to its reflog; this prevents two simultaneous writers from
clobbering each other's reflog lines (it does not even have
to be two symref updates; because we don't hold the lock, we
could race with somebody writing to the pointed-to ref via
HEAD, for example).
We can fix this by writing the reflog before we commit the
lockfile. This runs the risk of writing the reflog but
failing the final rename(), but at least we now err on the
same side as the rest of the ref code.
Noticed-by: Michael Haggerty [off-list ref]
Signed-off-by: Jeff King <redacted>
---
refs/files-backend.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
From: Jeff King <hidden> Date: 2016-06-15 23:07:34
The create_symref() function predates the existence of
"struct lock_file", let alone the more recent "struct
ref_lock". Instead, it just does its own manual dot-locking.
Besides being more code, this has a few downsides:
- if git is interrupted while holding the lock, we don't
clean up the lockfile
- we don't do the usual directory/filename conflict check.
So you can sometimes create a symref "refs/heads/foo/bar",
even if "refs/heads/foo" exists (namely, if the refs are
packed and we do not hit the d/f conflict in the
filesystem).
This patch refactors create_symref() to use the "struct
ref_lock" interface, which handles both of these things.
There are a few bonus cleanups that come along with it:
- we leaked ref_path in some error cases
- the symref contents were stored in a fixed-size buffer,
putting an artificial (albeit large) limitation on the
length of the refname. We now write through fprintf, and
handle refnames of any size.
- we called adjust_shared_perm only after the file was
renamed into place, creating a potential race with
readers in a shared repository. The lockfile code now
handles this when creating the lockfile, making it
atomic.
- the legacy prefer_symlink_refs path did not do any
locking at all. Admittedly, it is not atomic from a
reader's perspective (as it unlinks and re-creates the
symlink to overwrite), but at least it cannot conflict
with other writers now.
- the result of this patch is hopefully more readable. It
eliminates three goto labels. Two were for error checking
that is now simplified, and the third was to reach shared
code that has been pulled into its own function.
Signed-off-by: Jeff King <redacted>
---
refs/files-backend.c | 109 ++++++++++++++++++++++++------------------------
t/t1401-symbolic-ref.sh | 8 ++++
2 files changed, 62 insertions(+), 55 deletions(-)
@@ -2811,74 +2811,73 @@ static int commit_ref_update(struct ref_lock *lock,return0;}-intcreate_symref(constchar*refname,constchar*target,constchar*logmsg)+staticintcreate_ref_symlink(structref_lock*lock,constchar*target){-char*lockpath=NULL;-charbuf[1000];-intfd,len,written;-char*ref_path=git_pathdup("%s",refname);-unsignedcharold_sha1[20],new_sha1[20];-structstrbuferr=STRBUF_INIT;--if(logmsg&&read_ref(refname,old_sha1))-hashclr(old_sha1);--if(safe_create_leading_directories(ref_path)<0)-returnerror("unable to create directory for %s",ref_path);-+intret=-1;#ifndef NO_SYMLINK_HEAD-if(prefer_symlink_refs){-unlink(ref_path);-if(!symlink(target,ref_path))-gotodone;+char*ref_path=get_locked_file_path(lock->lk);+unlink(ref_path);+ret=symlink(target,ref_path);+free(ref_path);++if(ret)fprintf(stderr,"no symlink - falling back to symbolic ref\n");-}#endif+returnret;+}-len=snprintf(buf,sizeof(buf),"ref: %s\n",target);-if(sizeof(buf)<=len){-error("refname too long: %s",target);-gotoerror_free_return;-}-lockpath=mkpathdup("%s.lock",ref_path);-fd=open(lockpath,O_CREAT|O_EXCL|O_WRONLY,0666);-if(fd<0){-error("Unable to open %s for writing",lockpath);-gotoerror_free_return;-}-written=write_in_full(fd,buf,len);-if(close(fd)!=0||written!=len){-error("Unable to write to %s",lockpath);-gotoerror_unlink_return;-}-if(rename(lockpath,ref_path)<0){-error("Unable to create %s",ref_path);-gotoerror_unlink_return;-}-if(adjust_shared_perm(ref_path)){-error("Unable to fix permissions on %s",lockpath);-error_unlink_return:-unlink_or_warn(lockpath);-error_free_return:-free(lockpath);-free(ref_path);-return-1;-}-free(lockpath);--#ifndef NO_SYMLINK_HEAD-done:-#endif+staticvoidupdate_symref_reflog(structref_lock*lock,constchar*refname,+constchar*target,constchar*logmsg)+{+structstrbuferr=STRBUF_INIT;+unsignedcharnew_sha1[20];if(logmsg&&!read_ref(target,new_sha1)&&-log_ref_write(refname,old_sha1,new_sha1,logmsg,0,&err)){+log_ref_write(refname,lock->old_oid.hash,new_sha1,logmsg,0,&err)){error("%s",err.buf);strbuf_release(&err);}+}-free(ref_path);+staticintcreate_symref_locked(structref_lock*lock,constchar*refname,+constchar*target,constchar*logmsg)+{+if(prefer_symlink_refs&&!create_ref_symlink(lock,target)){+update_symref_reflog(lock,refname,target,logmsg);+return0;+}++if(!fdopen_lock_file(lock->lk,"w"))+returnerror("unable to fdopen %s: %s",+lock->lk->tempfile.filename.buf,strerror(errno));++/* no error check; commit_ref will check ferror */+fprintf(lock->lk->tempfile.fp,"ref: %s\n",target);+if(commit_ref(lock)<0)+returnerror("unable to write symref for %s: %s",refname,+strerror(errno));+update_symref_reflog(lock,refname,target,logmsg);return0;}+intcreate_symref(constchar*refname,constchar*target,constchar*logmsg)+{+structstrbuferr=STRBUF_INIT;+structref_lock*lock;+intret;++lock=lock_ref_sha1_basic(refname,NULL,NULL,NULL,REF_NODEREF,NULL,+&err);+if(!lock){+error("%s",err.buf);+strbuf_release(&err);+return-1;+}++ret=create_symref_locked(lock,refname,target,logmsg);+unlock_ref(lock);+returnret;+}+intreflog_exists(constchar*refname){structstatst;
From: Jeff King <hidden> Date: 2016-06-15 23:07:35
I'm iffy on this one, just because it drops a user-visible feature with
no deprecation. For the reasons given below, I doubt anybody is using it
for the original intended purpose, but maybe there is some crazy person
whose workflow revolves around core.preferSymlinkRefs.
A conservative choice would probably be to issue a deprecation warning
when we see it defined, wait a few versions, and then apply the patch
below.
-- >8 --
Long ago, in 2fabd21 (Disable USE_SYMLINK_HEAD by default,
2005-11-15), we switched git's default behavior to writing
text symrefs instead of symbolic links. Any scripts
accustomed to looking directly at .git/HEAD were updated to
use `rev-parse` instead. The Linux kernel's setlocalversion
script was one, and it was fixed in 117a93d (kbuild: Use git
in scripts/setlocalversion, 2006-01-04).
However, the problem still happened when bisecting the
kernel; pre-117a93d kernels would not build properly with a
newer git, because they wanted to look directly at HEAD. To
solve this, we added 9f0bb90 (core.prefersymlinkrefs: use
symlinks for .git/HEAD, 2006-05-02), which lets the user
turn on the old behavior, theoretically letting you bisect
older kernel history.
But there are a few complications with this solution:
- packed-refs means you are limited in what you can do
with .git/HEAD. If it is a symlink, you may `readlink`
it to see where it points, but you cannot necessarily
`cat .git/HEAD` to get the sha1, as the pointed-to ref
may be packed.
In particular, this means that the pre-117a93d kbuild
script would sometimes work and sometimes not.
- These days, we bisect on a detached HEAD. So during
bisection, .git/HEAD _is_ a regular file with the
sha1, and it works to `cat` it, whether or not you set
core.preferSymlinkRefs.
Such scripts will all be broken again if we move to
alternate ref backends. They should have learned to use
`rev-parse` long ago, and it is only bisecting ancient
history that is a problem.
Now that almost ten years have passed, it seems less likely
that developers will bisect so far back in history. And
moreover, this is but one of many possible problems
developers run into when trying to build versions. The
standard solution is to apply a "fixup" patch or other
workaround while test-building the project, and that would
work here, too.
This patch therefore drops core.preferSymlinkRefs
completely. There are a few reasons to want to do so:
1. It drops some code that is probably exercised very
rarely.
2. The symlink code is not up to the same standards as the
rest of the ref code. In particular, it is racy with
respect to simultaneous readers and writers.
3. If we want to eventually drop the symlink-reading code,
this is a good first step to deprecating it.
Signed-off-by: Jeff King <redacted>
---
Documentation/config.txt | 6 ------
cache.h | 1 -
config.c | 5 -----
contrib/completion/git-completion.bash | 1 -
environment.c | 1 -
refs/files-backend.c | 20 --------------------
t/t7201-co.sh | 12 ------------
t/t9903-bash-prompt.sh | 9 ---------
8 files changed, 55 deletions(-)
@@ -433,12 +433,6 @@ CIFS/Microsoft Windows. + False by default.-core.preferSymlinkRefs::- Instead of the default "symref" format for HEAD- and other symbolic reference files, use symbolic links.- This is sometimes needed to work with old scripts that- expect HEAD to be a symbolic link.- core.bare:: If true this repository is assumed to be 'bare' and has no working directory associated with it. If this is the case a
@@ -427,18 +427,6 @@ test_expect_success 'checkout w/--track from tag fails' 'test"z$(gitrev-parsemaster^0)"="z$(gitrev-parseHEAD)"'-test_expect_success'detach a symbolic link HEAD''-gitcheckoutmaster&&-gitconfig--boolcore.prefersymlinkrefsyes&&-gitcheckoutside&&-gitcheckoutmaster&&-it=$(gitsymbolic-refHEAD)&&-test"z$it"=zrefs/heads/master&&-here=$(gitrev-parse--verifyrefs/heads/master)&&-gitcheckoutside^&&-test"z$(gitrev-parse--verifyrefs/heads/master)"="z$here"-'- test_expect_success\'checkout with --track fakes a sensible -b <name>''gitconfigremote.origin.fetch"+refs/heads/*:refs/remotes/origin/*"&&
@@ -113,9 +113,6 @@ all::## Define NO_SYS_SELECT_H if you don't have sys/select.h.#-# Define NO_SYMLINK_HEAD if you never want .git/HEAD to be a symbolic link.-# Enable it on Windows. By default, symrefs are still used.-## Define NO_SVN_TESTS if you want to skip time-consuming SVN interoperability# tests. These tests take up a significant amount of the total test time# but are not needed unless you plan to talk to SVN repos.
@@ -1096,9 +1096,6 @@ GIT_CONF_SUBST([HAVE_BSD_SYSCTL]) # Define USE_PIC if you need the main git objects to be built with -fPIC # in order to build and link perl/Git.so. x86-64 seems to need this. #-# Define NO_SYMLINK_HEAD if you never want .git/HEAD to be a symbolic link.-# Enable it on Windows. By default, symrefs are still used.-# # Define NO_PTHREADS if we do not have pthreads. # # Define PTHREAD_LIBS to the linker flag used for Pthread support.
From: Michael Haggerty <hidden> Date: 2016-06-15 23:07:35
On 12/29/2015 06:55 AM, Jeff King wrote:
On Sun, Dec 20, 2015 at 02:26:37AM -0500, Jeff King wrote:
quoted
I noticed that an interrupt "git symbolic-ref" will not clean up
"HEAD.lock". So I started this series as an attempt to convert
create_symref() to "struct lock_file" to get the usual tempfile cleanup.
Here's version 2, based on comments from Michael. The first two patches
were picked out separately for jk/symbolic-ref-maint, so I've dropped
them here (so 1+2 here are the original 3+4).
The other differences from v1 are:
- use "refname" instead of "ref" to match surrounding code
- drop adjust_shared_perm, as lockfile does it for us
- adjust reflog writing order (done in a new patch)
The patches are:
[1/3]: create_symref: modernize variable names
[2/3]: create_symref: use existing ref-lock code
[3/3]: create_symref: write reflog while holding lock
Thanks, Peff. The whole series is
Reviewed-by: Michael Haggerty <redacted>
Michael
--
Michael Haggerty
mhagger@alum.mit.edu