From: Frank Lichtenheld <hidden> Date: 2016-06-15 22:43:00
Hi.
This patch series started with the thought that it is really cumbersome
to use the pserver access via git-cvsserver without giving the
nobody user write access to the .git directory itself (especially since
SQLite seems to insist on creating temporary files in the
same directory as the database itself on writes).
This problem itself is easily fixable with an one-line patch
to git-cvsserver that uses a gitcvs.dbdir configuration variable.
I tried to abstract the problem a bit more though and created
means to configure all aspects of the database backend in a
very flexible manner. I would glad about comments on wether
I made my solution overly complex on the way...
Most of the changes are tested intensively with test repositories,
exceptions are noted in the individual patches. More testing welcome
of course.
The documentation updates are not yet complete.
Gruesse,
Frank Lichtenheld
Documentation/git-cvsserver.txt | 12 ++++++
git-cvsserver.perl | 72 ++++++++++++++++++++++++++++------------
2 files changed, 63 insertions(+), 21 deletions(-)
@@ -91,7 +91,9 @@ $log->debug("Temporary directory is '$TEMP_DIR'");# if we are called with a pserver argument,# deal with the authentication cat before entering the# main loop+$state->{method}='ext';if(@ARGV&&$ARGV[0]eq'pserver'){+$state->{method}='pserver';my$line=<STDIN>;chomp$line;unless($lineeq'BEGIN AUTH REQUEST'){die"E Do not understand $line - expecting BEGIN AUTH REQUEST\n";
From: Frank Lichtenheld <hidden> Date: 2016-06-15 22:43:00
Make all the different parts of the database backend connection
configurable. This adds the following string configuration variables:
- gitcvs.dbdriver
- gitcvs.dbname
- gitcvs.dbuser
- gitcvs.dbpass
The default values emulate the current behavior exactly for
backwards compatibility.
All configuration variables can also be specified for a specific
access method (i.e. in the form gitcvs.<method>.<var>)
The dbdriver/dbuser/dbpass variables are added for completness.
No other backend than SQLite is tested yet.
The dbname variable on the other hand is useful with this backend
already (to not discriminate against other possible backends
it was not splitted in dbdir and dbfile).
Both dbname and dbuser support dynamic variable substitution where
the available variables are:
%m -- the CVS 'module' (i.e. GIT 'head') worked on
%a -- CVS access method used (i.e. 'ext' or 'pserver')
%u -- User name of the user invoking git-cvsserver
%G -- .git directory name
%g -- .git directory name, mangled to be used in a filename,
currently this substitutes all chars except for [\w.-]
with '_'
Signed-off-by: Frank Lichtenheld <redacted>
---
git-cvsserver.perl | 40 ++++++++++++++++++++++++++++++++++------
1 files changed, 34 insertions(+), 6 deletions(-)
@@ -2141,19 +2141,33 @@ sub newbless$self,$class;-$self->{dbdir}=$config."/";-die"Database dir '$self->{dbdir}' isn't a directory"unless(defined($self->{dbdir})and-d$self->{dbdir});-$self->{module}=$module;-$self->{file}=$self->{dbdir}."/gitcvs.$module.sqlite";-$self->{git_path}=$config."/";$self->{log}=$log;die"Git repo '$self->{git_path}' doesn't exist"unless(-d$self->{git_path});-$self->{dbh}=DBI->connect("dbi:SQLite:dbname=".$self->{file},"","");+$self->{dbdriver}=$cfg->{gitcvs}{$state->{method}}{dbdriver}||+$cfg->{gitcvs}{dbdriver}||"dbi:SQLite";+$self->{dbname}=$cfg->{gitcvs}{$state->{method}}{dbname}||+$cfg->{gitcvs}{dbname}||"%Ggitcvs.%m.sqlite";+$self->{dbuser}=$cfg->{gitcvs}{$state->{method}}{dbuser}||+$cfg->{gitcvs}{dbuser}||"";+$self->{dbpass}=$cfg->{gitcvs}{$state->{method}}{dbpass}||+$cfg->{gitcvs}{dbpass}||"";+my%mapping=(m=>$module,+a=>$state->{method},+u=>getlogin||getpwuid($<)||$<,+G=>$self->{git_path},+g=>mangle_dirname($self->{git_path}),+);+$self->{dbname}=~s/%([mauGg])/$mapping{$1}/eg;+$self->{dbuser}=~s/%([mauGg])/$mapping{$1}/eg;++$self->{dbh}=DBI->connect("$self->{dbdriver}:dbname=$self->{dbname}",+$self->{dbuser},+$self->{dbpass});$self->{tables}={};foreachmy$table($self->{dbh}->tables)
@@ -2857,5 +2871,19 @@ sub safe_pipe_capture {returnwantarray?@output:join('',@output);}+=head2mangle_dirname++createastringfromadirectorynamethatissuitabletouseas+partofafilename,mainlybyconvertingallcharsexcept\w.-to_++=cut+submangle_dirname{+my$dirname=shift;+returnunlessdefined$dirname;++$dirname=~s/[^\w.-]/_/g;++return$dirname;+}1;
From: Frank Lichtenheld <hidden> Date: 2016-06-15 22:43:00
Allow to override the gitcvs.enabled and gitcvs.logfile configuration
variables for each access method (i.e. "ext" or "pserver") in the
form gitcvs.<method>.<var>
Signed-off-by: Frank Lichtenheld <redacted>
---
Documentation/git-cvsserver.txt | 12 ++++++++++++
git-cvsserver.perl | 10 +++++++---
2 files changed, 19 insertions(+), 3 deletions(-)
@@ -68,6 +68,18 @@ Note: you need to ensure each user that is going to invoke git-cvsserver has write access to the log file and to the git repository. When offering anon access via pserver, this means that the nobody user should have write access to at least the sqlite database at the root of the repository.++Both configuration variables can also be overriden for a specific method of+access. Valid method names are "ext" (for SSH access) and "pserver". The+following example configuration would disable pserver access while still+allowing access over SSH.+------+ [gitcvs]+ enabled=0++ [gitcvs "ext"]+ enabled=1+------ -- 3. On the client machine you need to set the following variables. CVSROOT should be set as per normal, but the directory should point at the
@@ -191,7 +191,10 @@ sub req_Root}}-unless(defined($cfg->{gitcvs}{enabled})and$cfg->{gitcvs}{enabled}=~ /^\s*(1|true|yes)\s*$/i)+unless(($cfg->{gitcvs}{$state->{method}}{enabled}+and$cfg->{gitcvs}{$state->{method}}{enabled}=~ /^\s*(1|true|yes)\s*$/i)+or($cfg->{gitcvs}{enabled}+and$cfg->{gitcvs}{enabled}=~ /^\s*(1|true|yes)\s*$/i)){print"E GITCVS emulation needs to be enabled on this repo\n";print"E the repo config file needs a [gitcvs] section added, and the parameter 'enabled' set to 1\n";
@@ -200,9 +203,10 @@ sub req_Rootreturn0;}-if(defined($cfg->{gitcvs}{logfile}))+my$logfile=$cfg->{gitcvs}{$state->{method}}{logfile}||$cfg->{gitcvs}{logfile};+if($logfile){-$log->setfile($cfg->{gitcvs}{logfile});+$log->setfile($logfile);}else{$log->nofile();}
From: Frank Lichtenheld <hidden> Date: 2016-06-15 22:43:00
Currently all calls to the database backend make no
error checking or handling at all. At least abort
if the connection to the database failed since
there is really no way we could do anything useful
after that.
Signed-off-by: Frank Lichtenheld <redacted>
---
git-cvsserver.perl | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
@@ -2168,6 +2168,7 @@ sub new$self->{dbh}=DBI->connect("$self->{dbdriver}:dbname=$self->{dbname}",$self->{dbuser},$self->{dbpass});+die"Error connecting to database\n"unlessdefined$self->{dbh};$self->{tables}={};foreachmy$table($self->{dbh}->tables)
From: Frank Lichtenheld <hidden> Date: 2016-06-15 22:43:00
This is intended to be used in the form gitcvs.<method>.<var>
but this patch doesn't introduce any users yet.
Signed-off-by: Frank Lichtenheld <redacted>
---
git-cvsserver.perl | 8 ++++++--
1 files changed, 6 insertions(+), 2 deletions(-)
From: Martin Langhoff <hidden> Date: 2016-06-15 22:43:00
On 3/20/07, Frank Lichtenheld [off-list ref] wrote:
Make all the different parts of the database backend connection
configurable. This adds the following string configuration variables:
Nice. I guess the hard part of this is going to be creating DB schemas
that are reasonably portable. The SQL we use is as vanilla as it gets
;-)
Both dbname and dbuser support dynamic variable substitution where
the available variables are:
%m -- the CVS 'module' (i.e. GIT 'head') worked on
%a -- CVS access method used (i.e. 'ext' or 'pserver')
%u -- User name of the user invoking git-cvsserver
%G -- .git directory name
%g -- .git directory name, mangled to be used in a filename,
currently this substitutes all chars except for [\w.-]
with '_'
It's missing from the POD though ;-)
Good to see patches coming to cvsserver -- I haven't been able to do
much on it lately, and my pet projects are pretty hard. If anyone
cares, they are:
- mimic CVS branch support
- allow skewing version numbers to match an existing repo
with those 2 in place, we'd have a means of applying a "vampire tap"
to an existing cvs server and take over without anyone noticing. But
tehy are both hard, hard hard.
cheers.
martin
From: Frank Lichtenheld <hidden> Date: 2016-06-15 22:43:01
On Tue, Mar 20, 2007 at 07:47:12AM +1200, Martin Langhoff wrote:
On 3/20/07, Frank Lichtenheld [off-list ref] wrote:
quoted
Make all the different parts of the database backend connection
configurable. This adds the following string configuration variables:
Nice. I guess the hard part of this is going to be creating DB schemas
that are reasonably portable. The SQL we use is as vanilla as it gets
;-)
I've now actually made a quick test to see how we do when using other
backends (with PostgreSQL 8.2, will also do one with MySQL later).
Some problems that I saw:
- It would probably cool to be able to tell git-cvsserver that it
should use only one database for all modules (i.e. git branches)
This way one doesn't need to give the users database creation
privileges. Of course pre-creating all databases possibly ever needed
is possible but somewhat cumbersome.
- DBI->tables seems to be a portability problem. e.g. with SQLite
it returns "head", "commitmsgs", etc; with PostgreSQL it returns
public.head, public.commitmsgs, etc. The output of MySQL might
be different, too.
quoted
Both dbname and dbuser support dynamic variable substitution where
the available variables are:
%m -- the CVS 'module' (i.e. GIT 'head') worked on
%a -- CVS access method used (i.e. 'ext' or 'pserver')
%u -- User name of the user invoking git-cvsserver
%G -- .git directory name
%g -- .git directory name, mangled to be used in a filename,
currently this substitutes all chars except for [\w.-]
with '_'
It's missing from the POD though ;-)
You mean the asciidoc, right?
And yeah, I know. Writing English documentation is not actually one of my
preferred occupations :/ Will do it, though.
[...]
with those 2 in place, we'd have a means of applying a "vampire tap"
to an existing cvs server and take over without anyone noticing. But
tehy are both hard, hard hard.
From: Frank Lichtenheld <hidden> Date: 2016-06-15 22:43:01
On Fri, Mar 23, 2007 at 04:17:58PM +0100, Frank Lichtenheld wrote:
On Tue, Mar 20, 2007 at 07:47:12AM +1200, Martin Langhoff wrote:
quoted
On 3/20/07, Frank Lichtenheld [off-list ref] wrote:
quoted
Make all the different parts of the database backend connection
configurable. This adds the following string configuration variables:
Nice. I guess the hard part of this is going to be creating DB schemas
that are reasonably portable. The SQL we use is as vanilla as it gets
;-)
I've now actually made a quick test to see how we do when using other
backends (with PostgreSQL 8.2, will also do one with MySQL later).
Done the MySQL tests, too.
Some problems that I saw:
- It would probably cool to be able to tell git-cvsserver that it
should use only one database for all modules (i.e. git branches)
This way one doesn't need to give the users database creation
privileges. Of course pre-creating all databases possibly ever needed
is possible but somewhat cumbersome.
- DBI->tables seems to be a portability problem. e.g. with SQLite
it returns "head", "commitmsgs", etc; with PostgreSQL it returns
public.head, public.commitmsgs, etc. The output of MySQL might
be different, too.
It is `head`, and `revision`. Fun ;)
Why no etc.? Because pretty much every other used SQL command (than these
two "create table") fails with syntax errors. Not that I actually expected
anything else from MySQL...
Gruesse,
--
Frank Lichtenheld [off-list ref]
www: http://www.djpig.de/
From: Frank Lichtenheld <hidden> Date: 2016-06-15 22:43:02
DBI->table_info is portable across different DBD backends,
DBI->tables is not.
Limit the output to objects of type TABLE.
---
git-cvsserver.perl | 4 +---
1 files changed, 1 insertions(+), 3 deletions(-)
Obviously to be applied on top of my previous
patch series.
With this patch I was able to use DBD::Pg as backend.
It is not very comfortable because of the "one db for
each module" problem, but at least it works.
@@ -2171,10 +2171,8 @@ sub newdie"Error connecting to database\n"unlessdefined$self->{dbh};$self->{tables}={};-foreachmy$table($self->{dbh}->tables)+foreachmy$table(keys%{$self->{dbh}->table_info(undef,undef,undef,'TABLE')->fetchall_hashref('TABLE_NAME')}){-$table=~s/^"//;-$table=~s/"$//;$self->{tables}{$table}=1;}
From: Frank Lichtenheld <hidden> Date: 2016-06-15 22:43:02
On Sat, Mar 31, 2007 at 03:57:47PM +0200, Frank Lichtenheld wrote:
DBI->table_info is portable across different DBD backends,
DBI->tables is not.
Limit the output to objects of type TABLE.
---
I just noticed I forgot to add the
"Signed-off-by: Frank Lichtenheld [off-list ref]"
Does one resend the patch in such cases?
Gruesse,
--
Frank Lichtenheld [off-list ref]
www: http://www.djpig.de/
From: Frank Lichtenheld <hidden> Date: 2016-06-15 22:43:03
This patch series consists mainly of documentation updates for the
new features I introduced in my previous series. With all these
patches applied I would consider this topic in a shape so that it
could be included in an official release. It still contains
some experimental stuff but since the user has to enable that himself
and I put a "here be dragons" in the documentation I see no problem
with that.
Proofreading of patch 3 by someone that actually speaks English would
be greatly appreciated.
Documentation/git-cvsserver.txt | 97 +++++++++++++++++++++++++++++++++++-----
git-cvsserver.perl | 7 +-
2 files changed, 91 insertions(+), 13 deletions(-)
From: Frank Lichtenheld <hidden> Date: 2016-06-15 22:43:03
Documents the new configuration variables and the variable
substitution mechanism.
Signed-off-by: Frank Lichtenheld <redacted>
---
Documentation/git-cvsserver.txt | 87 +++++++++++++++++++++++++++++++++++++--
1 files changed, 83 insertions(+), 4 deletions(-)
Proofreading would be greatly appreciated.
@@ -65,11 +65,12 @@ env variable, you can rename git-cvsserver to cvs. ------ Note: you need to ensure each user that is going to invoke git-cvsserver has-write access to the log file and to the git repository. When offering anon-access via pserver, this means that the nobody user should have write access-to at least the sqlite database at the root of the repository.+write access to the log file and to the database (see+<<dbbackend,Database Backend>>. If you want to offer write access over+SSH, the users of course also need write access to the git repository itself.-Both configuration variables can also be overriden for a specific method of+[[configaccessmethod]]+All configuration variables can also be overriden for a specific method of access. Valid method names are "ext" (for SSH access) and "pserver". The following example configuration would disable pserver access while still allowing access over SSH.
@@ -105,6 +106,84 @@ Example: cvs co -d project-master master ------+[[dbbackend]]+Database Backend+----------------++git-cvsserver uses one database per git head (i.e. CVS module) to+store information about the repository for faster access. The+database doesn't contain any persitent data and can be completly+regenerated from the git repository at any time. The database+needs to be updated (i.e. written to) after every commit. That+means that even if you offer only read access (e.g. by using+the pserver method), git-cvsserver should have write access to+the database to work reliably (otherwise you need to make sure+that the database if up-to-date all the time git-cvsserver is run).++By default it uses SQLite databases in the git directory, named+`gitcvs.<module_name>.sqlite`. Note that the SQLite backend creates+temporary files in the same directory as the database file on+write so it might not be enough to grant the users using+git-cvsserver write access to the database file without granting+them also write access to the directory.++You can configure the database backend with the following+configuration variables:++Configuring database backend+~~~~~~~~~~~~~~~~~~~~~~~~~~~~++git-cvsserver uses the Perl DBI module. Please also read+its documentation if changing these variables, especially+about `DBI->connect()`.++gitcvs.dbname::+ Database name. The exact meaning depends on the+ used database driver, for SQLite this is a filename.+ Supports variable substitution (see below). May+ not contain semicolons (`;`).+ Default: '%Ggitcvs.%m.sqlite'++gitcvs.dbdriver::+ Used DBI driver. You can specify any available driver+ for this here, but it might not work. cvsserver is tested+ with 'DBD::SQLite', reported to work with+ 'DBD::Pg', and reported *not* to work with 'DBD::mysql'.+ Please regard this as an experimental feature. May not+ contain double colons (`:`).+ Default: 'SQLite'++gitcvs.dbuser::+ Database user. Only useful if setting `dbdriver`, since+ SQLite has no concept of database users. Supports variable+ substitution (see below).++gitcvs.dbpass::+ Database password. Only useful if setting `dbdriver`, since+ SQLite has no concept of database passwords.++All variables can also be set per access method, see <<configaccessmethod,above>>.++Variable substitution+^^^^^^^^^^^^^^^^^^^^^+In `dbdriver` and `dbuser` you can use the following variables:++%G::+ git directory name+%g::+ git directory name, where all characters except for+ alpha-numeric ones, `.`, and `-` are replaced with+ `_` (this should make it easier to use the directory+ name in a filename if wanted)+%m::+ CVS module/git head name+%a::+ access method (one of "ext" or "pserver")+%u::+ Name of the user running git-cvsserver.+ If no name can be determined, the+ numeric uid is used.+ Eclipse CVS Client Notes ------------------------
From: Frank Lichtenheld <hidden> Date: 2016-06-15 22:43:03
Fix a typo: s/Not/Note/
Some formating fixes: Use ` ` syntax for all filenames and
' ' syntax for all commandline switches.
Signed-off-by: Frank Lichtenheld <redacted>
---
Documentation/git-cvsserver.txt | 8 ++++----
1 files changed, 4 insertions(+), 4 deletions(-)
@@ -122,12 +122,12 @@ To get a checkout with the Eclipse CVS client: Protocol notes: If you are using anonymous access via pserver, just select that. Those using SSH access should choose the 'ext' protocol, and configure 'ext' access on the Preferences->Team->CVS->ExtConnection pane. Set CVS_SERVER to-'git-cvsserver'. Not that password support is not good when using 'ext',+'git-cvsserver'. Note that password support is not good when using 'ext', you will definitely want to have SSH keys setup. Alternatively, you can just use the non-standard extssh protocol that Eclipse offer. In that case CVS_SERVER is ignored, and you will have to replace-the cvs utility on the server with git-cvsserver or manipulate your .bashrc+the cvs utility on the server with git-cvsserver or manipulate your `.bashrc` so that calling 'cvs' effectively calls git-cvsserver. Clients known to work
@@ -146,9 +146,9 @@ checkout, diff, status, update, log, add, remove, commit. Legacy monitoring operations are not supported (edit, watch and related). Exports and tagging (tags and branches) are not supported at this stage.-The server should set the -k mode to binary when relevant, however,+The server should set the '-k' mode to binary when relevant, however, this is not really implemented yet. For now, you can force the server-to set `-kb` for all files by setting the `gitcvs.allbinary` config+to set '-kb' for all files by setting the `gitcvs.allbinary` config variable. In proper GIT tradition, the contents of the files are always respected. No keyword expansion or newline munging is supported.
From: Frank Lichtenheld <hidden> Date: 2016-06-15 22:43:03
Don't include the scheme name in gitcvs.dbdriver, it is
always 'dbi' anyway.
Don't allow ':' in driver names nor ';' in database names for
sanity reasons.
Signed-off-by: Frank Lichtenheld <redacted>
---
git-cvsserver.perl | 6 ++++--
1 files changed, 4 insertions(+), 2 deletions(-)
I wasn't sure whether I should send this as a new patch or as a new version of my
older one?
@@ -2149,7 +2149,7 @@ sub newdie"Git repo '$self->{git_path}' doesn't exist"unless(-d$self->{git_path});$self->{dbdriver}=$cfg->{gitcvs}{$state->{method}}{dbdriver}||-$cfg->{gitcvs}{dbdriver}||"dbi:SQLite";+$cfg->{gitcvs}{dbdriver}||"SQLite";$self->{dbname}=$cfg->{gitcvs}{$state->{method}}{dbname}||$cfg->{gitcvs}{dbname}||"%Ggitcvs.%m.sqlite";$self->{dbuser}=$cfg->{gitcvs}{$state->{method}}{dbuser}||
@@ -2165,7 +2165,9 @@ sub new$self->{dbname}=~s/%([mauGg])/$mapping{$1}/eg;$self->{dbuser}=~s/%([mauGg])/$mapping{$1}/eg;-$self->{dbh}=DBI->connect("$self->{dbdriver}:dbname=$self->{dbname}",+die"Invalid char ':' in dbdriver"if$self->{dbdriver}=~ /:/;+die"Invalid char ';' in dbname"if$self->{dbname}=~ /;/;+$self->{dbh}=DBI->connect("dbi:$self->{dbdriver}:dbname=$self->{dbname}",$self->{dbuser},$self->{dbpass});die"Error connecting to database\n"unlessdefined$self->{dbh};