Re: [RFC 1/4 v2] Implement a basic remote helper for svn in C.

On Thursday 26 July 2012 04:08:42 Jonathan Nieder wrote:

Florian Achleitner wrote:

quoted

On Monday 02 July 2012 06:07:41 Jonathan Nieder wrote:

[...]

quoted

quoted

quoted

+
+static inline void printd(const char* fmt, ...)

[...]

quoted

quoted

Why not use trace_printf and avoid the complication?

Hm.. I tried. It wasn't exactly what I wanted. When I use trace_printf,
it's activated together with all other traces. I can use trace_vprintf
and specify a key, but I would always have to print the header "rhsvn
debug: " and the key by hand. So I could replace vfprintf in this
function by trace_vprintf to do that. But then there's not much
simplification. (?)

Hmm.  There's no trace_printf_with_key() but that's presumably because
no one has needed it.  If it existed, you could use

	#define printd(msg) trace_printf_with_key("GIT_TRACE_REMOTE_SVN", "%s",
msg)

But now that I check, I don't see how the current printd() calls would
be useful to other people.  Why announce these moments and not others?
They're just temporary debugging cruft, right?

For that, plain trace_printf() works great.

Yes, it's for debugging only, I could just delete it all. It's inspired by 
transport-helper.c. The env var GIT_TRANSPORT_HELPER_DEBUG enables it. While 
transport-helper has a lot of if (debug) fprintf(..), I encapsulated it in 
printd.
So I should kick printd out?

quoted

quoted

quoted

+
+	printf("import\n");
+	printf("\n");
+	fflush(stdout);
+	return SUCCESS;
+}

Why the multiple printf?  Is the flush needed?

Excess printf gone.
Flush is needed. Otherwise it doesn't flush and the other end waits
forever.

Ah, fast-import is ready, remote helper is ready, no one initiates
pumping of data between them.  Maybe the purpose of the flush would
be more obvious if it were moved to the caller.

Acutally this goes to the git parent process (not fast-import), waiting for a 
reply to the command. I think I have to call flush on this side of the pipe. 
Can you flush it from the reader? This wouldn't have the desired effect, it 
drops buffered data.

[...]

quoted

quoted

quoted

+	/* opening a fifo for usually reading blocks until a writer has opened
it too. +	 * Therefore, we open with RDWR.
+	 */
+	report_fd = open(back_pipe_env, O_RDWR);
+	if(report_fd < 0) {
+		die("Unable to open fast-import back-pipe! %s", strerror(errno));
+	}

Is this necessary?  Why shouldn't we fork the writer first and wait
for it here?

Yes, necessary.

Oh, dear.  I hope not.  E.g., Cygwin doesn't support opening fifos
RDWR (out of scope for the gsoc project, but still).

I believe it can be solved using RDONLY and WRONLY too. Probably we solve it 
by not using the fifo at all.
Currently the blocking comes from the fact, that fast-import doesn't parse 
it's command line at startup. It rather reads an input line first and decides 
whether to parse the argv after reading the first input line or at the end of 
the input. (don't know why)
remote-svn opens the pipe before sending the first command to fast-import and 
blocks on the open, while fast-import waits for input --> deadlock.
with remote-svn: RDWR, fast-import: WRONLY, this works.

Other scenario: Nothing to import, remote-svn only sends 'done' and closes the 
pipe again. After fast-import reads the first line it parses it's command line 
and tries to open the fifo which is already closed on the other side --> 
blocks.
This is solved by using RDWR on both sides.

If we change the points where the pipes are openend and closed, this could be 
circumvented.

[...]

quoted

E.g. If there's have nothing to import, the helper sends only 'done' to
fast- import and quits.

Won't the writer open the pipe and wait for us to open our end before
doing that?

[...]

quoted

quoted

quoted

+
+	code = start_command(&svndump_proc);
+	if(code)
+		die("Unable to start %s, code %d", svndump_proc.argv[0], code);

start_command() is supposed to have printed a message already when it
fails, unless errno == ENOENT and silent_exec_failure was set.

Yes, but it doesn't die, right?

You can exit without writing a message with exit(), e.g. like so:

	if (code)
		exit(code);

or like so:

	if (code)
		exit(128);

ok, why not..

[...]

quoted

quoted

quoted

+
+	close(svndump_proc.out);

Important?  Wouldn't finish_command do this?

As far as I understood it, it doesn't close extra created pipes. Probably
I
just didn't find it in the code ..

So this is to work around a bug in the run-command interface?

Good question. 

[...]

quoted

quoted

quoted

+	close(report_fd);

What is the purpose of this step?

Close the back-report pipe end of the remote-helper.

That's just repeating the question. :)  Perhaps it's supposed to
trigger some action on the other end of the pipe?  It would just be
useful to add a comment documenting why one shouldn't remove this
close() call, or else will probably end up removing it and needlessly
suffering.

It's just closing files I've openend before. I usually do that, when i no 
longer need them. Isn't that common?
I believe it makes no difference if you only call import once. On subsequent 
imports the pipe is still open when it tries to open it again, that could be a 
problem. It would have to statically store report_fd, but what for??

[...]

quoted

quoted

quoted

+
+	code = finish_command(&svndump_proc);
+	if(code)
+		warning("Something went wrong with termination of %s, code %d",
svndump_proc.argv[0], code);

finish_command() is supposed to print a message when it fails.

I changed the message text. It should tell us if svnrdump exited with non-
zero.

I'd suggest looking at other finish_command() callers for examples.

[...]

quoted

quoted

quoted

+enum cmd_result do_command(struct strbuf* line)
+{
+	const command* p = command_list;
+	enum cmd_result ret;
+	printd("command line '%s'", line->buf);
+	while(*p) {
+		ret = (*p)(line);
+		if(ret != NOT_HANDLED)
+			return ret;
+		p++;
+	}

If possible, matching commands by name (like git.c does) would make
the behavior easier to predict.

There is some usecase for this. The intention was, that command handlers
should be able to process more than one 'name'. E.g. an import batch is
terminated by a newline. This newline is handled by the import handler if
it is a batch. (This feature wasn't implemented in the version reviewed
here.)

So I decided to let the handler functions tell if they handle this line.

[continued from (*) above]
... it isn't needed at the moment.

See http://c2.com/xp/YouArentGonnaNeedIt.html

Hm.. all three values are used, they're not for the future but for now.
Of course it could be done somehow else.

[...]

quoted

quoted

quoted

+	free((void*)url);
+	free((void*)private_refs);

Won't this crash?

Crash? It frees detached strbuf buffers.

I see "url = argv[2];" a few lines above, but it looks like the variable
was reused for two different purposes. :(

Ooops. That was introduced by splitting out "[RFC 07/16] Allow reading svn 
dumps from files via file:// urls." It makes url malloced unconditionally. Will 
fix.

I'm not sure why you detach url, by the way.  If the goal is to free it,
why not use strbuf_release()?

I used the strbuf because of it's nice formatting functions. But on the other 
hand, e.g. url_decode returns a char *, so I stored it in this type.
It doesn't make a big difference if I use three strbufs or one and detach and 
free the char * at the end. In fact, char * will consume some bytes less than 
a strbuf ;)
Another question is, if calling free just before exit makes sense anyways.
(But you learn it in programming courses, and it pleases valgrind :D )

Thanks,
Jonathan

Florian