[PATCH 0/4] git replace --edit --raw

DORMANTno replies

7 messages, 2 authors, 2016-06-15 · open the first message on its own page

[PATCH 0/4] git replace --edit --raw

From: Jeff King <hidden>
Date: 2016-06-15 23:01:44

I tried to use "git replace --edit" today to repair a corrupted tree
that had an empty filename:

  $ git replace --edit HEAD^{tree}
  fatal: empty filename in tree entry
  fatal: cat-file reported failure

Oops. One of the major purposes of the command is to repair broken
objects. This series introduces a "--raw" option to let you easily edit
the binary if need be.

The first two patches are unrelated cleanups I noticed in the area. The
third is refactoring, and the final one is the interesting bit.

There's a minor textual conflict in the documentation with
cc/replace-graft, but it's pretty straightforward to resolve.

  [1/4]: replace: replace spaces with tabs in indentation
  [2/4]: avoid double close of descriptors handed to run_command
  [3/4]: replace: use argv_array in export_object
  [4/4]: replace: add a --raw mode for --edit

-Peff

[PATCH 1/4] replace: replace spaces with tabs in indentation

From: Jeff King <hidden>
Date: 2016-06-15 23:01:44

This matches our usual style and the surrounding code.

Signed-off-by: Jeff King <redacted>
---
 builtin/replace.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/builtin/replace.c b/builtin/replace.c
index 1bb491d..8507835 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -23,9 +23,9 @@ static const char * const git_replace_usage[] = {
 };
 
 enum replace_format {
-      REPLACE_FORMAT_SHORT,
-      REPLACE_FORMAT_MEDIUM,
-      REPLACE_FORMAT_LONG
+	REPLACE_FORMAT_SHORT,
+	REPLACE_FORMAT_MEDIUM,
+	REPLACE_FORMAT_LONG
 };
 
 struct show_data {
-- 
2.0.0.566.gfe3e6b2

[PATCH 2/4] avoid double close of descriptors handed to run_command

From: Jeff King <hidden>
Date: 2016-06-15 23:01:44

When a file descriptor is given to run_command via the
"in", "out", or "err" parameters, run_command takes
ownership. The descriptor will be closed in the parent
process whether the process is spawned successfully or not,
and closing it again is wrong.

In practice this has not caused problems, because we usually
close() right after start_command returns, meaning no other
code has opened a descriptor in the meantime. So we just get
EBADF and ignore it (rather than accidentally closing
somebody else's descriptor!).

Signed-off-by: Jeff King <redacted>
---
I noticed the one in replace, and grepped around for other instances. I
think this is all of them. The only possible cases I didn't investigate
carefully were in send-pack/receive-pack. The logic there is quite
complicated, and I remember looking at and fixing close() issues there
not too long ago (e.g., 49ecfa1), so I assumed it was OK.

 builtin/replace.c | 2 --
 daemon.c          | 1 -
 2 files changed, 3 deletions(-)
diff --git a/builtin/replace.c b/builtin/replace.c
index 8507835..eb1d2ec 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -207,8 +207,6 @@ static void export_object(const unsigned char *sha1, const char *filename)
 
 	if (run_command(&cmd))
 		die("cat-file reported failure");
-
-	close(fd);
 }
 
 /*
diff --git a/daemon.c b/daemon.c
index f9c63e9..e1e424d 100644
--- a/daemon.c
+++ b/daemon.c
@@ -775,7 +775,6 @@ static void handle(int incoming, struct sockaddr *addr, socklen_t addrlen)
 		logerror("unable to fork");
 	else
 		add_child(&cld, addr, addrlen);
-	close(incoming);
 }
 
 static void child_handler(int signo)
-- 
2.0.0.566.gfe3e6b2

[PATCH 3/4] replace: use argv_array in export_object

From: Jeff King <hidden>
Date: 2016-06-15 23:01:44

This is a little more verbose, but will make it easier to
make parts of our command-line conditional (without
resorting to magic numbers or lots of NULLs to get an
appropriately sized argv array).

Signed-off-by: Jeff King <redacted>
---
 builtin/replace.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/builtin/replace.c b/builtin/replace.c
index eb1d2ec..2584170 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -193,15 +193,17 @@ static int replace_object(const char *object_ref, const char *replace_ref, int f
  */
 static void export_object(const unsigned char *sha1, const char *filename)
 {
-	const char *argv[] = { "--no-replace-objects", "cat-file", "-p", NULL, NULL };
-	struct child_process cmd = { argv };
+	struct child_process cmd = { NULL };
 	int fd;
 
 	fd = open(filename, O_WRONLY | O_CREAT | O_TRUNC, 0666);
 	if (fd < 0)
 		die_errno("unable to open %s for writing", filename);
 
-	argv[3] = sha1_to_hex(sha1);
+	argv_array_push(&cmd.args, "--no-replace-objects");
+	argv_array_push(&cmd.args, "cat-file");
+	argv_array_push(&cmd.args, "-p");
+	argv_array_push(&cmd.args, sha1_to_hex(sha1));
 	cmd.git_cmd = 1;
 	cmd.out = fd;
 
-- 
2.0.0.566.gfe3e6b2

[PATCH 4/4] replace: add a --raw mode for --edit

From: Jeff King <hidden>
Date: 2016-06-15 23:01:44

One of the purposes of "git replace --edit" is to help a
user repair objects which are malformed or corrupted.
Usually we pretty-print trees with "ls-tree", which is much
easier to work with than the raw binary data.  However, some
forms of corruption break the tree-walker, in which case our
pretty-printing fails, rendering "--edit" useless for the
user.

This patch introduces a "--raw" option, which lets you edit
the binary data in these instances.

Signed-off-by: Jeff King <redacted>
---
 Documentation/git-replace.txt |  8 ++++++++
 builtin/replace.c             | 31 +++++++++++++++++++++----------
 2 files changed, 29 insertions(+), 10 deletions(-)
diff --git a/Documentation/git-replace.txt b/Documentation/git-replace.txt
index 61461b9..089dcac 100644
--- a/Documentation/git-replace.txt
+++ b/Documentation/git-replace.txt
@@ -73,6 +73,14 @@ OPTIONS
 	newly created object. See linkgit:git-var[1] for details about
 	how the editor will be chosen.
 
+--raw::
+	When editing, provide the raw object contents rather than
+	pretty-printed ones. Currently this only affects trees, which
+	will be shown in their binary form. This is harder to work with,
+	but can help when repairing a tree that is so corrupted it
+	cannot be pretty-printed. Note that you may need to configure
+	your editor to cleanly read and write binary data.
+
 -l <pattern>::
 --list <pattern>::
 	List replace refs for objects that match the given pattern (or
diff --git a/builtin/replace.c b/builtin/replace.c
index 2584170..d1ea2c2 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -188,10 +188,12 @@ static int replace_object(const char *object_ref, const char *replace_ref, int f
 }
 
 /*
- * Write the contents of the object named by "sha1" to the file "filename",
- * pretty-printed for human editing based on its type.
+ * Write the contents of the object named by "sha1" to the file "filename".
+ * If "raw" is true, then the object's raw contents are printed according to
+ * "type". Otherwise, we pretty-print the contents for human editing.
  */
-static void export_object(const unsigned char *sha1, const char *filename)
+static void export_object(const unsigned char *sha1, enum object_type type,
+			  int raw, const char *filename)
 {
 	struct child_process cmd = { NULL };
 	int fd;
@@ -202,7 +204,10 @@ static void export_object(const unsigned char *sha1, const char *filename)
 
 	argv_array_push(&cmd.args, "--no-replace-objects");
 	argv_array_push(&cmd.args, "cat-file");
-	argv_array_push(&cmd.args, "-p");
+	if (raw)
+		argv_array_push(&cmd.args, typename(type));
+	else
+		argv_array_push(&cmd.args, "-p");
 	argv_array_push(&cmd.args, sha1_to_hex(sha1));
 	cmd.git_cmd = 1;
 	cmd.out = fd;
@@ -217,7 +222,7 @@ static void export_object(const unsigned char *sha1, const char *filename)
  * The sha1 of the written object is returned via sha1.
  */
 static void import_object(unsigned char *sha1, enum object_type type,
-			  const char *filename)
+			  int raw, const char *filename)
 {
 	int fd;
 
@@ -225,7 +230,7 @@ static void import_object(unsigned char *sha1, enum object_type type,
 	if (fd < 0)
 		die_errno("unable to open %s for reading", filename);
 
-	if (type == OBJ_TREE) {
+	if (!raw && type == OBJ_TREE) {
 		const char *argv[] = { "mktree", NULL };
 		struct child_process cmd = { argv };
 		struct strbuf result = STRBUF_INIT;
@@ -265,7 +270,7 @@ static void import_object(unsigned char *sha1, enum object_type type,
 	 */
 }
 
-static int edit_and_replace(const char *object_ref, int force)
+static int edit_and_replace(const char *object_ref, int force, int raw)
 {
 	char *tmpfile = git_pathdup("REPLACE_EDITOBJ");
 	enum object_type type;
@@ -281,10 +286,10 @@ static int edit_and_replace(const char *object_ref, int force)
 
 	check_ref_valid(old, prev, ref, sizeof(ref), force);
 
-	export_object(old, tmpfile);
+	export_object(old, type, raw, tmpfile);
 	if (launch_editor(tmpfile, NULL, NULL) < 0)
 		die("editing object file failed");
-	import_object(new, type, tmpfile);
+	import_object(new, type, raw, tmpfile);
 
 	free(tmpfile);
 
@@ -297,6 +302,7 @@ static int edit_and_replace(const char *object_ref, int force)
 int cmd_replace(int argc, const char **argv, const char *prefix)
 {
 	int force = 0;
+	int raw = 0;
 	const char *format = NULL;
 	enum {
 		MODE_UNSPECIFIED = 0,
@@ -310,6 +316,7 @@ int cmd_replace(int argc, const char **argv, const char *prefix)
 		OPT_CMDMODE('d', "delete", &cmdmode, N_("delete replace refs"), MODE_DELETE),
 		OPT_CMDMODE('e', "edit", &cmdmode, N_("edit existing object"), MODE_EDIT),
 		OPT_BOOL('f', "force", &force, N_("replace the ref if it exists")),
+		OPT_BOOL(0, "raw", &raw, N_("do not pretty-print contents for --edit")),
 		OPT_STRING(0, "format", &format, N_("format"), N_("use this format")),
 		OPT_END()
 	};
@@ -329,6 +336,10 @@ int cmd_replace(int argc, const char **argv, const char *prefix)
 		usage_msg_opt("-f only makes sense when writing a replacement",
 			      git_replace_usage, options);
 
+	if (raw && cmdmode != MODE_EDIT)
+		usage_msg_opt("--raw only makes sense with --edit",
+			      git_replace_usage, options);
+
 	switch (cmdmode) {
 	case MODE_DELETE:
 		if (argc < 1)
@@ -346,7 +357,7 @@ int cmd_replace(int argc, const char **argv, const char *prefix)
 		if (argc != 1)
 			usage_msg_opt("-e needs exactly one argument",
 				      git_replace_usage, options);
-		return edit_and_replace(argv[0], force);
+		return edit_and_replace(argv[0], force, raw);
 
 	case MODE_LIST:
 		if (argc > 1)
-- 
2.0.0.566.gfe3e6b2

Re: [PATCH 4/4] replace: add a --raw mode for --edit

From: Eric Sunshine <hidden>
Date: 2016-06-15 23:01:44

On Tue, Jun 24, 2014 at 5:46 AM, Jeff King [off-list ref] wrote:
One of the purposes of "git replace --edit" is to help a
user repair objects which are malformed or corrupted.
Usually we pretty-print trees with "ls-tree", which is much
easier to work with than the raw binary data.  However, some
forms of corruption break the tree-walker, in which case our
pretty-printing fails, rendering "--edit" useless for the
user.

This patch introduces a "--raw" option, which lets you edit
the binary data in these instances.
Is there a possibility that any of the other git-replace modes will
grow a need for "raw"? If not, would it make sense to make this
specific to "edit" as --edit=raw?
quoted hunk
Signed-off-by: Jeff King <redacted>
---
 Documentation/git-replace.txt |  8 ++++++++
 builtin/replace.c             | 31 +++++++++++++++++++++----------
 2 files changed, 29 insertions(+), 10 deletions(-)
diff --git a/Documentation/git-replace.txt b/Documentation/git-replace.txt
index 61461b9..089dcac 100644
--- a/Documentation/git-replace.txt
+++ b/Documentation/git-replace.txt
@@ -73,6 +73,14 @@ OPTIONS
        newly created object. See linkgit:git-var[1] for details about
        how the editor will be chosen.

+--raw::
+       When editing, provide the raw object contents rather than
+       pretty-printed ones. Currently this only affects trees, which
+       will be shown in their binary form. This is harder to work with,
+       but can help when repairing a tree that is so corrupted it
+       cannot be pretty-printed. Note that you may need to configure
+       your editor to cleanly read and write binary data.
+
 -l <pattern>::
 --list <pattern>::
        List replace refs for objects that match the given pattern (or
diff --git a/builtin/replace.c b/builtin/replace.c
index 2584170..d1ea2c2 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -188,10 +188,12 @@ static int replace_object(const char *object_ref, const char *replace_ref, int f
 }

 /*
- * Write the contents of the object named by "sha1" to the file "filename",
- * pretty-printed for human editing based on its type.
+ * Write the contents of the object named by "sha1" to the file "filename".
+ * If "raw" is true, then the object's raw contents are printed according to
+ * "type". Otherwise, we pretty-print the contents for human editing.
  */
-static void export_object(const unsigned char *sha1, const char *filename)
+static void export_object(const unsigned char *sha1, enum object_type type,
+                         int raw, const char *filename)
 {
        struct child_process cmd = { NULL };
        int fd;
@@ -202,7 +204,10 @@ static void export_object(const unsigned char *sha1, const char *filename)

        argv_array_push(&cmd.args, "--no-replace-objects");
        argv_array_push(&cmd.args, "cat-file");
-       argv_array_push(&cmd.args, "-p");
+       if (raw)
+               argv_array_push(&cmd.args, typename(type));
+       else
+               argv_array_push(&cmd.args, "-p");
        argv_array_push(&cmd.args, sha1_to_hex(sha1));
        cmd.git_cmd = 1;
        cmd.out = fd;
@@ -217,7 +222,7 @@ static void export_object(const unsigned char *sha1, const char *filename)
  * The sha1 of the written object is returned via sha1.
  */
 static void import_object(unsigned char *sha1, enum object_type type,
-                         const char *filename)
+                         int raw, const char *filename)
 {
        int fd;
@@ -225,7 +230,7 @@ static void import_object(unsigned char *sha1, enum object_type type,
        if (fd < 0)
                die_errno("unable to open %s for reading", filename);

-       if (type == OBJ_TREE) {
+       if (!raw && type == OBJ_TREE) {
                const char *argv[] = { "mktree", NULL };
                struct child_process cmd = { argv };
                struct strbuf result = STRBUF_INIT;
@@ -265,7 +270,7 @@ static void import_object(unsigned char *sha1, enum object_type type,
         */
 }

-static int edit_and_replace(const char *object_ref, int force)
+static int edit_and_replace(const char *object_ref, int force, int raw)
 {
        char *tmpfile = git_pathdup("REPLACE_EDITOBJ");
        enum object_type type;
@@ -281,10 +286,10 @@ static int edit_and_replace(const char *object_ref, int force)

        check_ref_valid(old, prev, ref, sizeof(ref), force);

-       export_object(old, tmpfile);
+       export_object(old, type, raw, tmpfile);
        if (launch_editor(tmpfile, NULL, NULL) < 0)
                die("editing object file failed");
-       import_object(new, type, tmpfile);
+       import_object(new, type, raw, tmpfile);

        free(tmpfile);
@@ -297,6 +302,7 @@ static int edit_and_replace(const char *object_ref, int force)
 int cmd_replace(int argc, const char **argv, const char *prefix)
 {
        int force = 0;
+       int raw = 0;
        const char *format = NULL;
        enum {
                MODE_UNSPECIFIED = 0,
@@ -310,6 +316,7 @@ int cmd_replace(int argc, const char **argv, const char *prefix)
                OPT_CMDMODE('d', "delete", &cmdmode, N_("delete replace refs"), MODE_DELETE),
                OPT_CMDMODE('e', "edit", &cmdmode, N_("edit existing object"), MODE_EDIT),
                OPT_BOOL('f', "force", &force, N_("replace the ref if it exists")),
+               OPT_BOOL(0, "raw", &raw, N_("do not pretty-print contents for --edit")),
                OPT_STRING(0, "format", &format, N_("format"), N_("use this format")),
                OPT_END()
        };
@@ -329,6 +336,10 @@ int cmd_replace(int argc, const char **argv, const char *prefix)
                usage_msg_opt("-f only makes sense when writing a replacement",
                              git_replace_usage, options);

+       if (raw && cmdmode != MODE_EDIT)
+               usage_msg_opt("--raw only makes sense with --edit",
+                             git_replace_usage, options);
+
        switch (cmdmode) {
        case MODE_DELETE:
                if (argc < 1)
@@ -346,7 +357,7 @@ int cmd_replace(int argc, const char **argv, const char *prefix)
                if (argc != 1)
                        usage_msg_opt("-e needs exactly one argument",
                                      git_replace_usage, options);
-               return edit_and_replace(argv[0], force);
+               return edit_and_replace(argv[0], force, raw);

        case MODE_LIST:
                if (argc > 1)
--
2.0.0.566.gfe3e6b2
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 4/4] replace: add a --raw mode for --edit

From: Jeff King <hidden>
Date: 2016-06-15 23:01:44

On Tue, Jun 24, 2014 at 09:40:09PM -0400, Eric Sunshine wrote:
On Tue, Jun 24, 2014 at 5:46 AM, Jeff King [off-list ref] wrote:
quoted
One of the purposes of "git replace --edit" is to help a
user repair objects which are malformed or corrupted.
Usually we pretty-print trees with "ls-tree", which is much
easier to work with than the raw binary data.  However, some
forms of corruption break the tree-walker, in which case our
pretty-printing fails, rendering "--edit" useless for the
user.

This patch introduces a "--raw" option, which lets you edit
the binary data in these instances.
Is there a possibility that any of the other git-replace modes will
grow a need for "raw"? If not, would it make sense to make this
specific to "edit" as --edit=raw?
I doubt that any other modes will want it, as it is about the
pretty-printing step which is pretty specific to --edit. However, making
it "--edit=raw" also precludes adding other "modes" to --edit. I do
not have any in mind, but I do not see it as impossible.

Preclude is maybe a strong word. You could have "--edit=raw,flag1,flag2",
but then we are essentially reinventing an option parser inside --edit's
value. Not to mention that you cannot do "--no-raw", even without other
flags being added.

-Peff
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help