Re: [PATCH] Introduce a filter-path argument to git-daemon, for doing custom path transformations

Johan Sørensen [off-list ref] writes:

quoted

Do you run git-daemon from inetd, or standalone, by the way?

Standalone.

quoted

I am wondering how well it would scale if you spawn an external "filter path"
script every time you get a request.

A quick test of 250 consecutive requests with ls-remote to localhost
(all without the --verbose flag), slowest run:
- Baseline (no --filter-path agument): 3.39s

$ cat filter.c
#import "stdio.h"
int main (int argc, char const *argv[]) {
	printf("%s", "/existing.git\0");
	return 0;
}
- 3.84s

$ cat filter.rb
#!/usr/bin/ruby
print "/existing.git\0"
- 4.76s

So, obviously highly dependent on how long it takes the script to
launch and how much work it does. And yes, neither of the above really
does anything :) nor takes any increased cpu load into account

Another approach is to keep the external script running and feed it on
stdin, but that would involve a bit more micro-management of the
external process. I will revisit that idea if I find out that's
needed.

I actually was hoping (especially we have Dscho on Cc: list) that somebody
like you would start suggesting a "plug in" approach to load .so files,
which would lead to a easy-to-port dso support with the help from msysgit
folks we can use later in other parts of the system (e.g. customizable
filters used for diff textconv, clean/smudge, etc.)

quoted

(by the way, "filter path" sounds as if it checks and conditionally
denies access to, or something like that, which is not what you are using
it for.  It is more about rewriting paths, a la mod_rewrite, and I think
the option is misnamed)

Maybe --rewrite-script or --rewrite-command  instead?

Perhaps.

Hi,

On Sat, 14 Mar 2009, Junio C Hamano wrote:

Johan Sørensen [off-list ref] writes:

quoted

quoted

Do you run git-daemon from inetd, or standalone, by the way?

Standalone.

quoted

I am wondering how well it would scale if you spawn an external 
"filter path" script every time you get a request.

A quick test of 250 consecutive requests with ls-remote to localhost
(all without the --verbose flag), slowest run:
- Baseline (no --filter-path agument): 3.39s

$ cat filter.c
#import "stdio.h"
int main (int argc, char const *argv[]) {
	printf("%s", "/existing.git\0");
	return 0;
}
- 3.84s

$ cat filter.rb
#!/usr/bin/ruby
print "/existing.git\0"
- 4.76s

So, obviously highly dependent on how long it takes the script to 
launch and how much work it does. And yes, neither of the above really 
does anything :) nor takes any increased cpu load into account

Another approach is to keep the external script running and feed it on 
stdin, but that would involve a bit more micro-management of the 
external process. I will revisit that idea if I find out that's 
needed.

I actually was hoping (especially we have Dscho on Cc: list) that somebody
like you would start suggesting a "plug in" approach to load .so files,
which would lead to a easy-to-port dso support with the help from msysgit
folks we can use later in other parts of the system (e.g. customizable
filters used for diff textconv, clean/smudge, etc.)

I do not like that at all.  Dynamic libraries -- especially on Windows -- 
are a major hassle.

However, I cannot think of anything Johan might want to do that would not 
be possible using a bunch of regular expressions together with 
substitions.

FWIW I have experimental code in my personal tree that sports 
strbuf_regsub(), a function to replace matches of a regular expression 
(possibly with groups) by a given string (which may contain \0 .. \9, 
being replaced with the respective group's contents).

Ciao,
Dscho

2009/3/19 Johannes Schindelin [off-list ref]:

quoted

I actually was hoping (especially we have Dscho on Cc: list) that somebody
like you would start suggesting a "plug in" approach to load .so files,
which would lead to a easy-to-port dso support with the help from msysgit
folks we can use later in other parts of the system (e.g. customizable
filters used for diff textconv, clean/smudge, etc.)

I do not like that at all.  Dynamic libraries -- especially on Windows --
are a major hassle.

However, I cannot think of anything Johan might want to do that would not
be possible using a bunch of regular expressions together with
substitions.

Let me reiterate my use-case then: I need to dynamically substitute
one path with another. Perhaps "map" paints a better picture than
"substitute" here. Please refer to my second mail in this thread for
more details.

The only way I can see regexps work, is that if they're read, on a
per-request basis (reloading git-daemon every time they change is just
silly), from somewhere outside the git-daemon. Then, you might as well
take the full-on approach this patch provides.

Cheers,
JS

FWIW I have experimental code in my personal tree that sports
strbuf_regsub(), a function to replace matches of a regular expression
(possibly with groups) by a given string (which may contain \0 .. \9,
being replaced with the respective group's contents).

Ciao,
Dscho

Hi,

On Thu, 19 Mar 2009, Johan Sørensen wrote:

2009/3/19 Johannes Schindelin [off-list ref]:

quoted

quoted

I actually was hoping (especially we have Dscho on Cc: list) that 
somebody like you would start suggesting a "plug in" approach to load 
.so files, which would lead to a easy-to-port dso support with the 
help from msysgit folks we can use later in other parts of the system 
(e.g. customizable filters used for diff textconv, clean/smudge, 
etc.)

I do not like that at all.  Dynamic libraries -- especially on Windows 
-- are a major hassle.

However, I cannot think of anything Johan might want to do that would 
not be possible using a bunch of regular expressions together with 
substitions.

Let me reiterate my use-case then: I need to dynamically substitute one 
path with another. Perhaps "map" paints a better picture than 
"substitute" here. Please refer to my second mail in this thread for 
more details.

The only way I can see regexps work, is that if they're read, on a 
per-request basis (reloading git-daemon every time they change is just 
silly), from somewhere outside the git-daemon. Then, you might as well 
take the full-on approach this patch provides.

quoted

FWIW I have experimental code in my personal tree that sports 
strbuf_regsub(), a function to replace matches of a regular expression 
(possibly with groups) by a given string (which may contain \0 .. \9, 
being replaced with the respective group's contents).

Do not get me wrong, I can see your use case.

But I have been cautioning against other possibly regrettably things, and 
it gave me _no_ pleasure at all to be proven correct in hindsight.

I'd rather be called grumpy old Git, be ridiculed and insulted, but at the 
same time have precautions in git.git that prevent having to admit 
mournfully that some change was not so brilliant after all.

So if some rules consisting of regular expressions with appropriate 
substitutions, even if they will have to be updated from time to time, 
solve your case, I'd rather have that than allow a server to run external 
programs that are not exactly well audited against all kinds of attacks.

Ciao,
Dscho