From: Felipe Contreras <hidden> Date: 2016-06-15 22:58:33
Hi,
Here are the patches that allow transport helpers to be completely transparent;
renaming branches, deleting them, custom refspecs, --force, --dry-run,
reporting forced update, everything works.
Some of these were were sent before and rejected without a reason, but here
they are again in case anybody is interested.
Felipe Contreras (9):
transport-helper: add 'force' to 'export' helpers
transport-helper: check for 'forced update' message
fast-export: improve argument parsing
fast-export: add new --refspec option
transport-helper: add support for old:new refspec
fast-import: add support to delete refs
fast-export: add support to delete refs
transport-helper: add support to delete branches
transport-helper: don't update refs in dry-run
Documentation/git-fast-export.txt | 4 ++++
Documentation/git-fast-import.txt | 3 +++
builtin/fast-export.c | 47 ++++++++++++++++++++++++++++++++++++++-
fast-import.c | 13 ++++++++---
t/t5801-remote-helpers.sh | 10 ++++++++-
t/t9300-fast-import.sh | 18 +++++++++++++++
t/t9350-fast-export.sh | 18 +++++++++++++++
transport-helper.c | 42 ++++++++++++++++++++++++----------
8 files changed, 138 insertions(+), 17 deletions(-)
--
1.8.4-fc
From: Felipe Contreras <hidden> Date: 2016-06-15 22:58:33
Otherwise they cannot know when to force the push or not (other than
hacks).
Signed-off-by: Felipe Contreras <redacted>
---
transport-helper.c | 3 +++
1 file changed, 3 insertions(+)
@@ -814,6 +814,9 @@ static int push_refs_with_export(struct transport *transport,die("helper %s does not support dry-run",data->name);}+if(flags&TRANSPORT_PUSH_FORCE)+set_helper_option(transport,"force","true");+helper=get_helper(transport);write_constant(helper->in,"export\n");
From: Felipe Contreras <hidden> Date: 2016-06-15 22:58:33
So the remote-helpers can tell us when a forced push was needed.
Signed-off-by: Felipe Contreras <redacted>
---
transport-helper.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
From: Felipe Contreras <hidden> Date: 2016-06-15 22:58:33
We don't want to pass arguments specific to fast-export to
setup_revisions.
Signed-off-by: Felipe Contreras <redacted>
---
builtin/fast-export.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
@@ -105,6 +105,10 @@ marks the same across runs. in the commit (as opposed to just listing the files which are different from the commit's first parent).+--refspec::+ Apply the specified refspec to each ref exported. Multiple of them can+ be specified.+ [<git-rev-list-args>...]:: A list of arguments, acceptable to 'git rev-parse' and 'git rev-list', that specifies the specific objects and references
@@ -661,6 +673,7 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix)structcommit*commit;char*export_filename=NULL,*import_filename=NULL;uint32_tlastimportid;+structstring_listrefspecs_list;structoptionoptions[]={OPT_INTEGER(0,"progress",&progress,N_("show progress after <n> objects")),
@@ -681,6 +694,8 @@ int cmd_fast_export(int argc, const char **argv, const char *prefix)OPT_BOOLEAN(0,"use-done-feature",&use_done_feature,N_("Use the done feature to terminate the stream")),OPT_BOOL(0,"no-data",&no_data,N_("Skip output of blob data")),+OPT_STRING_LIST(0,"refspec",&refspecs_list,N_("refspec"),+N_("Apply refspec to exported refs")),OPT_END()};
@@ -504,4 +504,11 @@ test_expect_success 'refs are updated even if no commits need to be exported' 'test_cmpexpectedactual'+test_expect_success'use refspec''+gitfast-export--refspecrefs/heads/master:refs/heads/foobarmaster|\+grep"^commit "|sort|uniq>actual&&+echo"commit refs/heads/foobar">expected&&+test_cmpexpectedactual+'+ test_done
@@ -87,7 +87,7 @@ test_expect_success 'push new branch by name' 'compare_refslocalHEADserverrefs/heads/new-name'-test_expect_failure'push new branch with old:new refspec''+test_expect_success'push new branch with old:new refspec''(cdlocal&&gitpushoriginnew-name:new-refspec)&&
@@ -809,7 +809,7 @@ static int push_refs_with_export(struct transport *transport,structref*ref;structchild_process*helper,exporter;structhelper_data*data=transport->data;-structstring_listrevlist_args=STRING_LIST_INIT_NODUP;+structstring_listrevlist_args=STRING_LIST_INIT_DUP;structstrbufbuf=STRBUF_INIT;if(!data->refspecs)
@@ -848,8 +848,13 @@ static int push_refs_with_export(struct transport *transport,die("remote-helpers do not support ref deletion");if(ref->peer_ref){-if(strcmp(ref->peer_ref->name,ref->name))-die("remote-helpers do not support old:new syntax");+if(strcmp(ref->name,ref->peer_ref->name)){+structstrbufbuf=STRBUF_INIT;+strbuf_addf(&buf,"%s:%s",ref->peer_ref->name,ref->name);+string_list_append(&revlist_args,"--refspec");+string_list_append(&revlist_args,buf.buf);+strbuf_release(&buf);+}string_list_append(&revlist_args,ref->peer_ref->name);}}
@@ -857,6 +862,8 @@ static int push_refs_with_export(struct transport *transport,if(get_exporter(transport,&exporter,&revlist_args))die("Couldn't run fast-export");+string_list_clear(&revlist_args,1);+if(finish_command(&exporter))die("Error while running fast-export");push_update_refs_status(data,remote_refs);
@@ -483,6 +483,9 @@ Marks must be declared (via `mark`) before they can be used. * Any valid Git SHA-1 expression that resolves to a commit. See ``SPECIFYING REVISIONS'' in linkgit:gitrevisions[7] for details.+* The special null SHA-1 (40 zeros) specifices that the branch is to be+ removed.+ The special case of restarting an incremental import from the current branch value should be written as: ----
@@ -1674,10 +1675,13 @@ static int update_branch(struct branch *b)structref_lock*lock;unsignedcharold_sha1[20];-if(is_null_sha1(b->sha1))-return0;if(read_ref(b->name,old_sha1))hashclr(old_sha1);+if(is_null_sha1(b->sha1)){+if(b->delete)+delete_ref(b->name,old_sha1,0);+return0;+}lock=lock_any_ref_for_update(b->name,old_sha1,0);if(!lock)returnerror("Unable to lock %s",b->name);
@@ -2604,8 +2608,11 @@ static int parse_from(struct branch *b)free(buf);}elseparse_from_existing(b);-}elseif(!get_sha1(from,b->sha1))+}elseif(!get_sha1(from,b->sha1)){parse_from_existing(b);+if(is_null_sha1(b->sha1))+b->delete=1;+}elsedie("Invalid ref name or SHA1 expression: %s",from);
@@ -2934,4 +2934,22 @@ test_expect_success 'S: ls with garbage after sha1 must fail' 'test_i18ngrep"space after tree-ish"err'+test_expect_success'T: delete branch''+gitbranchto-delete&&+gitfast-import<<-EOF&&+resetrefs/heads/to-delete+from0000000000000000000000000000000000000000+EOF+test_must_failgitrev-parse--verifyrefs/heads/to-delete+'++test_expect_success'T: empty reset doesnt delete branch''+gitbranchnot-to-delete&&+gitfast-import<<-EOF&&+resetrefs/heads/not-to-delete+EOF+gitshow-ref&&+gitrev-parse--verifyrefs/heads/not-to-delete+'+ test_done
@@ -844,18 +844,19 @@ static int push_refs_with_export(struct transport *transport,}free(private);-if(ref->deletion)-die("remote-helpers do not support ref deletion");-if(ref->peer_ref){if(strcmp(ref->name,ref->peer_ref->name)){structstrbufbuf=STRBUF_INIT;-strbuf_addf(&buf,"%s:%s",ref->peer_ref->name,ref->name);+if(!ref->deletion)+strbuf_addf(&buf,"%s:%s",ref->peer_ref->name,ref->name);+else+strbuf_addf(&buf,":%s",ref->name);string_list_append(&revlist_args,"--refspec");string_list_append(&revlist_args,buf.buf);strbuf_release(&buf);}-string_list_append(&revlist_args,ref->peer_ref->name);+if(!ref->deletion)+string_list_append(&revlist_args,ref->peer_ref->name);}}
From: Felipe Contreras <hidden> Date: 2016-06-15 22:58:33
The remote helper namespace should not be updated.
Signed-off-by: Felipe Contreras <redacted>
---
transport-helper.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
@@ -715,7 +715,8 @@ static int push_update_ref_status(struct strbuf *buf,}staticvoidpush_update_refs_status(structhelper_data*data,-structref*remote_refs)+structref*remote_refs,+intflags){structstrbufbuf=STRBUF_INIT;structref*ref=remote_refs;
@@ -729,7 +730,7 @@ static void push_update_refs_status(struct helper_data *data,if(push_update_ref_status(&buf,&ref,remote_refs))continue;-if(!data->refspecs)+if(flags&TRANSPORT_PUSH_DRY_RUN||!data->refspecs)continue;/* propagate back the update to the remote namespace */
@@ -799,7 +800,7 @@ static int push_refs_with_push(struct transport *transport,sendline(data,&buf);strbuf_release(&buf);-push_update_refs_status(data,remote_refs);+push_update_refs_status(data,remote_refs,flags);return0;}
@@ -867,7 +868,7 @@ static int push_refs_with_export(struct transport *transport,if(finish_command(&exporter))die("Error while running fast-export");-push_update_refs_status(data,remote_refs);+push_update_refs_status(data,remote_refs,flags);return0;}
From: Richard Hansen <hidden> Date: 2016-06-15 22:58:52
On 2013-08-29 11:23, Felipe Contreras wrote:
Here are the patches that allow transport helpers to be completely transparent;
renaming branches, deleting them, custom refspecs, --force, --dry-run,
reporting forced update, everything works.
What is the status of these patches?
I would like to be able to 'git push old:new' and 'git push -f' to a
bzr repo, and these are a prerequisite. I imagine changes to
git-remote-bzr will also be required; I'm willing write up such
changes to git-remote-bzr, but only if there's interest in picking up
this patch series.
Thanks,
Richard
@@ -844,18 +844,19 @@ static int push_refs_with_export(struct transport *transport,}free(private);-if(ref->deletion)-die("remote-helpers do not support ref deletion");-
The above deleted lines actually appear twice in transport-helper.c due
to an incorrect merge conflict resolution in
99d9ec090677c925c534001f01cbaf303a31cb82. The other copy of those lines should
also be deleted:
@@ -874,9 +874,6 @@ static int push_refs_with_export(struct transport *transport,char*private;unsignedcharsha1[20];-if(ref->deletion)-die("remote-helpers do not support ref deletion");-private=apply_refspecs(data->refspecs,data->refspec_nr,ref->name);if(private&&!get_sha1(private,sha1)){strbuf_addf(&buf,"^%s",private);
@@ -105,6 +105,10 @@ marks the same across runs. in the commit (as opposed to just listing the files which are different from the commit's first parent).+--refspec::+ Apply the specified refspec to each ref exported. Multiple of them can+ be specified.+
Do you mean '--refspec=<refspec>' and/or '--refspec <refspec>'?
How are the multiple refspecs specified? Space/comma/colon separated
list? Or multiple '--refspec' arguments with one refspec per '--refspec'?
@@ -504,4 +504,11 @@ test_expect_success 'refs are updated even if no commits need to be exported' 'test_cmpexpectedactual'+test_expect_success'use refspec''+gitfast-export--refspecrefs/heads/master:refs/heads/foobarmaster|\+grep"^commit "|sort|uniq>actual&&+echo"commit refs/heads/foobar">expected&&+test_cmpexpectedactual+'+ test_done
I think it'd be good to add a test for multiple refspecs.
-Richard
From: Felipe Contreras <hidden> Date: 2016-06-15 22:58:52
Richard Hansen wrote:
On 2013-08-29 11:23, Felipe Contreras wrote:
quoted
Here are the patches that allow transport helpers to be completely transparent;
renaming branches, deleting them, custom refspecs, --force, --dry-run,
reporting forced update, everything works.
What is the status of these patches?
Ask Junio. All the valid feedback has been addressed, there's no reason not to
apply them.
I would like to be able to 'git push old:new' and 'git push -f' to a
bzr repo, and these are a prerequisite. I imagine changes to
git-remote-bzr will also be required; I'm willing write up such
changes to git-remote-bzr, but only if there's interest in picking up
this patch series.
There's no need to modify git-remote-bzr, if theese patches are applied,
old:new would work for all remote-helpers.
--
Felipe Contreras
@@ -844,18 +844,19 @@ static int push_refs_with_export(struct transport *transport,}free(private);-if(ref->deletion)-die("remote-helpers do not support ref deletion");-
The above deleted lines actually appear twice in transport-helper.c due
to an incorrect merge conflict resolution in
99d9ec090677c925c534001f01cbaf303a31cb82. The other copy of those lines should
also be deleted:
@@ -874,9 +874,6 @@ static int push_refs_with_export(struct transport *transport,char*private;unsignedcharsha1[20];-if(ref->deletion)-die("remote-helpers do not support ref deletion");-private=apply_refspecs(data->refspecs,data->refspec_nr,ref->name);if(private&&!get_sha1(private,sha1)){strbuf_addf(&buf,"^%s",private);
From: Richard Hansen <hidden> Date: 2016-06-15 22:58:52
On 2013-08-29 11:23, Felipe Contreras wrote:
quoted hunk
Otherwise they cannot know when to force the push or not (other than
hacks).
Signed-off-by: Felipe Contreras <redacted>
---
transport-helper.c | 3 +++
1 file changed, 3 insertions(+)
@@ -814,6 +814,9 @@ static int push_refs_with_export(struct transport *transport,die("helper %s does not support dry-run",data->name);}+if(flags&TRANSPORT_PUSH_FORCE)+set_helper_option(transport,"force","true");
Should the return value of set_helper_option() be checked?
Also, should there be a #define TRANS_OPT_FORCE "force" with
TRANS_OPT_FORCE added to boolean_options[]?
Thanks,
Richard
@@ -105,6 +105,10 @@ marks the same across runs. in the commit (as opposed to just listing the files which are different from the commit's first parent).+--refspec::+ Apply the specified refspec to each ref exported. Multiple of them can+ be specified.+
Do you mean '--refspec=<refspec>' and/or '--refspec <refspec>'?
@@ -504,4 +504,11 @@ test_expect_success 'refs are updated even if no commits need to be exported' 'test_cmpexpectedactual'+test_expect_success'use refspec''+gitfast-export--refspecrefs/heads/master:refs/heads/foobarmaster|\+grep"^commit "|sort|uniq>actual&&+echo"commit refs/heads/foobar">expected&&+test_cmpexpectedactual+'+ test_done
I think it'd be good to add a test for multiple refspecs.
Maybe. If these patches were to actually be applied.
--
Felipe Contreras
From: Felipe Contreras <hidden> Date: 2016-06-15 22:58:52
Richard Hansen wrote:
On 2013-08-29 11:23, Felipe Contreras wrote:
quoted
Otherwise they cannot know when to force the push or not (other than
hacks).
Signed-off-by: Felipe Contreras <redacted>
---
transport-helper.c | 3 +++
1 file changed, 3 insertions(+)
@@ -814,6 +814,9 @@ static int push_refs_with_export(struct transport *transport,die("helper %s does not support dry-run",data->name);}+if(flags&TRANSPORT_PUSH_FORCE)+set_helper_option(transport,"force","true");
Should the return value of set_helper_option() be checked?
Yeah, it would make sense. I guess we want to die() if the user does
'git push -f' and the remote helper doesn't support that.
Also, should there be a #define TRANS_OPT_FORCE "force" with
I don't see the point of that. Defines are useful when you want to change the
value string, so you don't have to change the string everywhere, but we
definitely don't want to do that, as that would break backwards compatibility,
so TRANS_OPT_KEEP would always be "keep" so it's just a way to tire our
fingers.
TRANS_OPT_FORCE added to boolean_options[]?
I don't see how that would help us, the only thing that would achieve is to map:
set_helper_option(transport, "force", 1);
to
set_helper_option(transport, "force", "true");
But we are already doing that.
Moreover, the code is already doing something similar for all the other options.
set_helper_option(t, "progress", t->progress ? "true" : "false");
set_helper_option(t, "verbosity", buf);
set_helper_option(transport, "servpath", exec);
set_helper_option(transport, "dry-run", "true");
--
Felipe Contreras