From: brian m. carlson <hidden> Date: 2020-06-19 22:52:04
I recently sent out part 2/3 of the SHA-256 work and then I realized
that I hadn't built with the CVS and SVN adapters. So this is part
2.5/3 (or, to be pedantic, 3/4).
The patches are ordered roughly with test-only fixes, then SVN, and then
CVS. There are no additional tests here because everything fails with
SHA-256 before and passes afterwards.
brian m. carlson (14):
t9109: make test hash independent
t9168: make test hash independent
t9108: make test hash independent
t9100: make test work with SHA-256
t9104: make hash size independent
t9101: make hash independent
t/lib-git-svn: make hash size independent
perl: create and switch variables for hash constants
perl: make Git::IndexInfo work with SHA-256
perl: make SVN code hash independent
git-svn: set the OID length based on hash algorithm
git-cvsserver: port to SHA-256
git-cvsimport: port to SHA-256
git-cvsexportcommit: port to SHA-256
git-cvsexportcommit.perl | 14 ++--
git-cvsimport.perl | 8 +--
git-cvsserver.perl | 37 +++++-----
git-svn.perl | 25 ++++---
perl/Git/IndexInfo.pm | 6 +-
perl/Git/SVN.pm | 83 +++++++++++-----------
perl/Git/SVN/Editor.pm | 8 +--
perl/Git/SVN/Fetcher.pm | 6 +-
perl/Git/SVN/Log.pm | 2 +-
perl/Git/SVN/Ra.pm | 4 +-
t/lib-git-svn.sh | 25 +++----
t/t9100-git-svn-basic.sh | 19 ++++-
t/t9101-git-svn-props.sh | 12 ++--
t/t9104-git-svn-follow-parent.sh | 3 +-
t/t9108-git-svn-glob.sh | 4 +-
t/t9109-git-svn-multi-glob.sh | 6 +-
t/t9168-git-svn-partially-globbed-names.sh | 8 +--
17 files changed, 154 insertions(+), 116 deletions(-)
From: brian m. carlson <hidden> Date: 2020-06-19 22:51:35
Compute the relevant tree objects for SHA-256 and use those when
appropriate instead of using the SHA-1 ones.
Signed-off-by: brian m. carlson <redacted>
---
t/t9100-git-svn-basic.sh | 19 ++++++++++++++++---
1 file changed, 16 insertions(+), 3 deletions(-)
From: brian m. carlson <hidden> Date: 2020-06-19 22:51:37
The size of a record in the database used by git svn is four bytes plus
the length of the binary hash. Instead of hard-coding 24, compute this
value based on the size of the hash in use.
Signed-off-by: brian m. carlson <redacted>
---
t/t9104-git-svn-follow-parent.sh | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
@@ -161,6 +161,7 @@ test_expect_success "track initial change if it was only made to parent" '' test_expect_success"follow-parent is atomic"'+record_size=$(($(test_oidrawsz)+4))&&(cdwc&&svn_cmdup&&
@@ -186,7 +187,7 @@ test_expect_success "follow-parent is atomic" 'mkdir-p"$GIT_DIR"/svn/refs/remotes/flunk@18&&rev_map=$(cd"$GIT_DIR"/svn/refs/remotes/stunk&&ls.rev_map*)&&ddif="$GIT_DIR"/svn/refs/remotes/stunk/$rev_map\-of="$GIT_DIR"/svn/refs/remotes/flunk@18/$rev_mapbs=24count=1&&+of="$GIT_DIR"/svn/refs/remotes/flunk@18/$rev_mapbs=$record_sizecount=1&&rm-rf"$GIT_DIR"/svn/refs/remotes/stunk&&gitsvninit--minimize-url-iflunk"$svnrepo"/flunk&&gitsvnfetch-iflunk&&
From: brian m. carlson <hidden> Date: 2020-06-19 22:51:38
Instead of hard-coding the object ID for our test .gitignore file, let's
compute it.
Signed-off-by: brian m. carlson <redacted>
---
t/t9101-git-svn-props.sh | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
From: brian m. carlson <hidden> Date: 2020-06-19 22:51:40
Most of the Git modules, git-svn excepted, don't know anything about the
hash algorithm and mostly work. However, when we're printing an
all-zero object ID in Git::IndexInfo, we need to know the hash length.
Since we don't want to change the API to have that information passed
in, let's query the config to find the hash algorithm and compute the
right value.
Signed-off-by: brian m. carlson <redacted>
---
perl/Git/IndexInfo.pm | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
From: brian m. carlson <hidden> Date: 2020-06-19 22:51:41
The record size used in the git svn storage is four bytes plus the
length of the binary hash. Pass the hash length into our Perl
invocation and use it to compute the size of the records.
Signed-off-by: brian m. carlson <redacted>
---
t/lib-git-svn.sh | 25 +++++++++++++------------
1 file changed, 13 insertions(+), 12 deletions(-)
From: brian m. carlson <hidden> Date: 2020-06-19 22:51:42
Instead of calling the function is_sha1, call it is_oid and update it to
match either a SHA-1 or a SHA-256 hex object ID.
Signed-off-by: brian m. carlson <redacted>
---
git-cvsimport.perl | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
@@ -810,7 +810,7 @@ ()open(my$fh,'-|',qw(git write-tree))ordie"unable to open git write-tree: $!";chomp(my$tree=<$fh>);-is_sha1($tree)+is_oid($tree)ordie"Cannot get tree id ($tree): $!";close($fh)ordie"Error running git write-tree: $?\n";
@@ -896,7 +896,7 @@ sub commit {print"Committed patch $patchset ($branch $commit_date)\n"if$opt_v;chomp(my$cid=<$commit_read>);-is_sha1($cid)ordie"Cannot get commit id ($cid): $!\n";+is_oid($cid)ordie"Cannot get commit id ($cid): $!\n";print"Commit ID $cid\n"if$opt_v;close($commit_read);
From: brian m. carlson <hidden> Date: 2020-06-19 22:51:43
When reading the configuration or when creating a new repository, load
the extensions.objectFormat value and set the object ID length to 64 if
it's "sha256". Note that we use the hex length in git-svn because most
of our processing is done on hex values, not binary ones.
Signed-off-by: brian m. carlson <redacted>
---
git-svn.perl | 7 +++++++
1 file changed, 7 insertions(+)
From: brian m. carlson <hidden> Date: 2020-06-19 22:51:45
When we apply a binary patch, we must have the full object ID in the
header in order to apply it; without that, any attempt to apply it will
fail. If we set GIT_DIR to empty, git apply does not know about the
hash algorithm we're using, and consequently any attempt to apply a
patch using SHA-256 will fail, since the object ID is the wrong length.
The reason we set the GIT_DIR environment variable is because we don't
want to modify the index; we just want to know whether the patch
applies. Instead, let's just use a temporary file for the index, which
will be cleaned up automatically when the object goes out of scope.
Additionally, read the configuration for the repository and compute the
length of an object ID based on it. Use that when matching object IDs
with a regex or computing the all-zeros object ID.
Signed-off-by: brian m. carlson <redacted>
---
git-cvsexportcommit.perl | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
@@ -22,6 +22,10 @@my$repo=Git->repository();$opt_w=$repo->config('cvsexportcommit.cvsdir')unlessdefined$opt_w;+my$tmpdir=File::Temp->newdir;+my$hash_algo=$repo->config('extensions.objectformat')||'sha1';+my$hexsz=$hash_algoeq'sha256'?64:40;+if($opt_w||$opt_W){# Remember where GIT_DIR is before changing to CVS checkoutunless($ENV{GIT_DIR}){
@@ -96,7 +100,7 @@}if($stageeq'headers'){-if($line=~m/^parent (\w{40})$/){# found a parent+if($line=~m/^parent ([0-9a-f]{$hexsz})$/){# found a parentpush@parents,$1;}elsif($line=~m/^author (.+) \d+ [-+]\d+$/){$author=$1;
@@ -111,7 +115,7 @@}}-my$noparent="0000000000000000000000000000000000000000";+my$noparent="0"x$hexsz;if($parent){my$found;# double check that it's a valid parent
@@ -174,7 +178,7 @@print"Checking if patch will apply\n";my@stat;-openAPPLY,"GIT_DIR= git-apply $context --summary --numstat<.cvsexportcommit.diff|"||die"cannot patch";+openAPPLY,"GIT_INDEX_FILE=$tmpdir/index git-apply $context --summary --numstat<.cvsexportcommit.diff|"||die"cannot patch";@stat=<APPLY>;closeAPPLY||die"Cannot patch";my(@bfiles,@files,@afiles,@dfiles);
@@ -407,7 +411,7 @@if($opt_W){system("git checkout $go_back_to")&&die"cannot move back to $go_back_to";-if(!($go_back_to=~ /^[0-9a-fA-F]{40}$/)){+if(!($go_back_to=~ /^[0-9a-fA-F]{$hexsz}$/)){system("git symbolic-ref HEAD $go_back_to")&&die"cannot move back to $go_back_to";}
From: brian m. carlson <hidden> Date: 2020-06-19 22:51:46
The code of git-cvsserver currently has several hard-coded 20 and 40
constants that are the length of SHA-1. When parsing the configuration
file, read the extensions.objectformat configuration setting as well as
CVS-related ones and adjust the hash sizes accordingly. Use these
computed values in all the places we match object IDs.
Signed-off-by: brian m. carlson <redacted>
---
git-cvsserver.perl | 37 ++++++++++++++++++++-----------------
1 file changed, 20 insertions(+), 17 deletions(-)
@@ -2375,7 +2378,7 @@ sub req_annotateprint"E ***************\n";while(<ANNOTATE>){-if(m/^([a-zA-Z0-9]{40})\t\([^\)]*\)(.*)$/i)+if(m/^([a-zA-Z0-9]{$state->{hexsz}})\t\([^\)]*\)(.*)$/i){my$commithash=$1;my$data=$2;
@@ -2852,7 +2855,7 @@ sub transmitfilereturn;}-die"Need filehash"unless(defined($filehash)and$filehash=~ /^[a-zA-Z0-9]{40}$/);+die"Need filehash"unless(defined($filehash)and$filehash=~ /^[a-zA-Z0-9]{$state->{hexsz}}$/);my$type=safe_pipe_capture('git','cat-file','-t',$filehash);chomp$type;
@@ -3042,7 +3045,7 @@ sub ensureWorkTreemy$ver=safe_pipe_capture('git','show-ref','-s',"refs/heads/$state->{module}");chomp$ver;-if($ver!~/^[0-9a-f]{40}$/)+if($ver!~/^[0-9a-f]{$state->{hexsz}}$/){$log->warn("Error from git show-ref -s refs/head$state->{module}");print"error 1 cannot find the current HEAD of module";
@@ -3281,7 +3284,7 @@ sub open_blob_or_die}elsif($srcTypeeq"sha1"){-unless(defined($name)and$name=~ /^[a-zA-Z0-9]{40}$/)+unless(defined($name)and$name=~ /^[a-zA-Z0-9]{$state->{hexsz}}$/){$log->warn("Need filehash");die"Need filehash\n";
@@ -3817,7 +3820,7 @@ sub updatechomp$commitsha1;my$commitinfo=::safe_pipe_capture('git','cat-file','commit',$self->{module});-unless($commitinfo=~ /tree\s+[a-zA-Z0-9]{40}/)+unless($commitinfo=~ /tree\s+[a-zA-Z0-9]{$state->{hexsz}}/){die("Invalid module '$self->{module}'");}
@@ -3957,7 +3960,7 @@ sub updatewhile(<FILELIST>){chomp;-unless( /^:\d{6}\s+([0-7]{6})\s+[a-f0-9]{40}\s+([a-f0-9]{40})\s+(\w)$/o)+unless( /^:\d{6}\s+([0-7]{6})\s+[a-f0-9]{$state->{hexsz}}\s+([a-f0-9]{$state->{hexsz}})\s+(\w)$/o){die("Couldn't process git-diff-tree line : $_");}
@@ -4625,11 +4628,11 @@ sub getmeta$db_query->execute($filename,$intRev);$meta=$db_query->fetchrow_hashref;}-elsif($revision=~ /^2\.1\.1\.2000(\.[1-3][0-9][0-9]){20}$/)+elsif($revision=~ /^2\.1\.1\.2000(\.[1-3][0-9][0-9]){$state->{rawsz}}$/){my($commitHash)=($revision=~/^2\.1\.1\.2000(.*)$/);$commitHash=~s/\.([0-9]+)/sprintf("%02x",$1-100)/eg;-if($commitHash=~/^[0-9a-f]{40}$/)+if($commitHash=~/^[0-9a-f]{$state->{hexsz}}$/){return$self->getMetaFromCommithash($filename,$commitHash);}
@@ -4639,7 +4642,7 @@ sub getmeta$log->warning("failed get $revision with commithash=$commitHash");undef$revision;}-elsif($revision=~ /^[0-9a-f]{40}$/)+elsif($revision=~ /^[0-9a-f]{$state->{hexsz}}$/){# Try DB first. This is mostly only useful for req_annotate(),# which only calls this for stuff that should already be in
@@ -4658,7 +4661,7 @@ sub getmetaif(!$meta){my($revCommit)=$self->lookupCommitRef($revision);-if($revCommit=~/^[0-9a-f]{40}$/)+if($revCommit=~/^[0-9a-f]{$state->{hexsz}}$/){return$self->getMetaFromCommithash($filename,$revCommit);}
@@ -4672,7 +4675,7 @@ sub getmetaelse{my($revCommit)=$self->lookupCommitRef($revision);-if($revCommit=~/^[0-9a-f]{40}$/)+if($revCommit=~/^[0-9a-f]{$state->{hexsz}}$/){return$self->getMetaFromCommithash($filename,$revCommit);}
@@ -4863,7 +4866,7 @@ sub lookupCommitRef$commitHash=::safe_pipe_capture("git","rev-parse","--verify","--quiet",$self->unescapeRefName($ref));$commitHash=~s/\s*$//;-if(!($commitHash=~/^[0-9a-f]{40}$/))+if(!($commitHash=~/^[0-9a-f]{$state->{hexsz}}$/)){$commitHash=undef;}
@@ -4909,7 +4912,7 @@ sub commitmessagemy$commithash=shift;my$tablename=$self->tablename("commitmsgs");-die("Need commithash")unless(defined($commithash)and$commithash=~ /^[a-zA-Z0-9]{40}$/);+die("Need commithash")unless(defined($commithash)and$commithash=~ /^[a-zA-Z0-9]{$state->{hexsz}}$/);my$db_query;$db_query=$self->{dbh}->prepare_cached("SELECT value FROM $tablename WHERE key=?",{},1);
From: brian m. carlson <hidden> Date: 2020-06-19 22:52:04
Instead of stripping off the first 41 characters of git log output,
let's just strip off the first space-separated component, which will
work for any size hash.
Signed-off-by: brian m. carlson <redacted>
---
t/t9109-git-svn-multi-glob.sh | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
From: brian m. carlson <hidden> Date: 2020-06-19 22:52:06
Instead of stripping off the first 41 characters of git log output,
let's just strip off the first space-separated component, which will
work for any size hash.
Signed-off-by: brian m. carlson <redacted>
---
t/t9108-git-svn-glob.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
From: brian m. carlson <hidden> Date: 2020-06-19 22:52:07
Instead of stripping off the first 41 characters of git log output,
let's just strip off the first space-separated component, which will
work for any size hash.
Signed-off-by: brian m. carlson <redacted>
---
t/t9168-git-svn-partially-globbed-names.sh | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
@@ -129,7 +129,7 @@ test_expect_success 'test prefixed globs match just prefix' 'test$(gitrev-parserefs/remotes/three/tags/t_~1)=\$(gitrev-parserefs/remotes/three/branches/b_)&&gitlog--pretty=onelinerefs/remotes/three/tags/t_>actual&&-sed-e"s/^.\{41\}//"actual>output.three&&+cut-d" "-f2-actual>output.three&&test_cmpexpect.threeoutput.three'
@@ -199,7 +199,7 @@ test_expect_success 'test globbing in the middle of the word' 'test$(gitrev-parserefs/remotes/five/tags/fghij~1)=\$(gitrev-parserefs/remotes/five/branches/abcde)&&gitlog--pretty=onelinerefs/remotes/five/tags/fghij>actual&&-sed-e"s/^.\{41\}//"actual>output.five&&+cut-d" "-f2-actual>output.five&&test_cmpexpect.fiveoutput.five'
From: brian m. carlson <hidden> Date: 2020-06-19 22:52:11
There are several places throughout git-svn that use various hard-coded
constants. For matching object IDs, use the $oid variable. Compute the
record size we use for our revision storage based on the object ID.
When parsing the revision map format, use a wildcard in the pack format
since we know that the data we're parsing is always exactly the record
size. This lets us continue to use a constant for the pack format.
Finally, update several comments to reflect the fact that an object ID
may be of one of multiple sizes.
Signed-off-by: brian m. carlson <redacted>
---
git-svn.perl | 4 +--
perl/Git/SVN.pm | 73 +++++++++++++++++++++--------------------
perl/Git/SVN/Editor.pm | 6 ++--
perl/Git/SVN/Fetcher.pm | 6 ++--
perl/Git/SVN/Ra.pm | 4 +--
5 files changed, 48 insertions(+), 45 deletions(-)
@@ -2087,10 +2087,10 @@ sub rebuild_from_rev_db {openmy$fh,'<',$pathorcroak"open: $!";binmode$fhorcroak"binmode: $!";while(<$fh>){-length($_)==41orcroak"inconsistent size in ($_) != 41";+length($_)==$::oid_length+1orcroak"inconsistent size in ($_)";chomp($_);++$r;-nextif$_eq('0'x40);+nextif$_eq('0'x$::oid_length);$self->rev_map_set($r,$_);print"r$r = $_\n";}
@@ -2196,9 +2196,9 @@ sub rebuild {# (mainly tags)## The format is this:-# - 24 bytes for every record,+# - 24 or 36 bytes for every record,# * 4 bytes for the integer representing an SVN revision number-# * 20 bytes representing the sha1 of a git commit+# * 20 or 32 bytes representing the oid of a git commit# - No empty padding records like the old format# (except the last record, which can be overwritten)# - new records are written append-only since SVN revision numbers
@@ -2207,7 +2207,7 @@ sub rebuild {# - Piping the file to xxd -c24 is a good way of dumping it for# viewing or editing (piped back through xxd -r), should the need# ever arise.-# - The last record can be padding revision with an all-zero sha1+# - The last record can be padding revision with an all-zero oid# This is used to optimize fetch performance when using multiple# "fetch" directives in .git/config#
@@ -2215,38 +2215,39 @@ sub rebuild {sub_rev_map_set{my($fh,$rev,$commit)=@_;+my$record_size=($::oid_length/2)+4;binmode$fhorcroak"binmode: $!";my$size=(stat($fh))[7];-($size%24)==0orcroak"inconsistent size: $size";+($size%$record_size)==0orcroak"inconsistent size: $size";my$wr_offset=0;if($size>0){-sysseek($fh,-24,SEEK_END)orcroak"seek: $!";-my$read=sysread($fh,my$buf,24)orcroak"read: $!";-$read==24orcroak"read only $read bytes (!= 24)";+sysseek($fh,-$record_size,SEEK_END)orcroak"seek: $!";+my$read=sysread($fh,my$buf,$record_size)orcroak"read: $!";+$read==$record_sizeorcroak"read only $read bytes (!= $record_size)";my($last_rev,$last_commit)=unpack(rev_map_fmt,$buf);-if($last_commiteq('0'x40)){-if($size>=48){-sysseek($fh,-48,SEEK_END)orcroak"seek: $!";-$read=sysread($fh,$buf,24)or+if($last_commiteq('0'x$::oid_length)){+if($size>=($record_size*2)){+sysseek($fh,-($record_size*2),SEEK_END)orcroak"seek: $!";+$read=sysread($fh,$buf,$record_size)orcroak"read: $!";-$read==24or-croak"read only $read bytes (!= 24)";+$read==$record_sizeor+croak"read only $read bytes (!= $record_size)";($last_rev,$last_commit)=unpack(rev_map_fmt,$buf);-if($last_commiteq('0'x40)){+if($last_commiteq('0'x$::oid_length)){croak"inconsistent .rev_map\n";}}if($last_rev>=$rev){croak"last_rev is higher!: $last_rev >= $rev";}-$wr_offset=-24;+$wr_offset=-$record_size;}}sysseek($fh,$wr_offset,SEEK_END)orcroak"seek: $!";-syswrite($fh,pack(rev_map_fmt,$rev,$commit),24)==24or+syswrite($fh,pack(rev_map_fmt,$rev,$commit),$record_size)==$record_sizeorcroak"write: $!";}
@@ -2271,7 +2272,7 @@ sub mkfile {subrev_map_set{my($self,$rev,$commit,$update_ref,$uuid)=@_;defined$commitordie"missing arg3\n";-length$commit==40ordie"arg3 must be a full SHA1 hexsum\n";+$commit=~/^$::oid$/ordie"arg3 must be a full hex object ID\n";my$db=$self->map_path($uuid);my$db_lock="$db.lock";my$sigmask;
@@ -2344,29 +2345,30 @@ sub rev_map_max {subrev_map_max_norebuild{my($self,$want_commit)=@_;+my$record_size=($::oid_length/2)+4;my$map_path=$self->map_path;stat$map_pathorreturn$want_commit?(0,undef):0;sysopen(my$fh,$map_path,O_RDONLY)orcroak"open: $!";binmode$fhorcroak"binmode: $!";my$size=(stat($fh))[7];-($size%24)==0orcroak"inconsistent size: $size";+($size%$record_size)==0orcroak"inconsistent size: $size";if($size==0){close$fhorcroak"close: $!";return$want_commit?(0,undef):0;}-sysseek($fh,-24,SEEK_END)orcroak"seek: $!";-sysread($fh,my$buf,24)==24orcroak"read: $!";+sysseek($fh,-$record_size,SEEK_END)orcroak"seek: $!";+sysread($fh,my$buf,$record_size)==$record_sizeorcroak"read: $!";my($r,$c)=unpack(rev_map_fmt,$buf);-if($want_commit&&$ceq('0'x40)){-if($size<48){+if($want_commit&&$ceq('0'x$::oid_length)){+if($size<$record_size*2){return$want_commit?(0,undef):0;}-sysseek($fh,-48,SEEK_END)orcroak"seek: $!";-sysread($fh,$buf,24)==24orcroak"read: $!";+sysseek($fh,-($record_size*2),SEEK_END)orcroak"seek: $!";+sysread($fh,$buf,$record_size)==$record_sizeorcroak"read: $!";($r,$c)=unpack(rev_map_fmt,$buf);-if($ceq('0'x40)){+if($ceq('0'x$::oid_length)){croak"Penultimate record is all-zeroes in $map_path";}}
@@ -400,12 +400,12 @@ sub T {($m->{mode_b}!~/^120/&&$m->{mode_a}=~/^120/)){$self->D({mode_a=>$m->{mode_a},mode_b=>'000000',-sha1_a=>$m->{sha1_a},sha1_b=>'0'x40,+sha1_a=>$m->{sha1_a},sha1_b=>'0'x$::oid_length,chg=>'D',file_b=>$m->{file_b}},$deletions);$self->A({mode_a=>'000000',mode_b=>$m->{mode_b},-sha1_a=>'0'x40,sha1_b=>$m->{sha1_b},+sha1_a=>'0'x$::oid_length,sha1_b=>$m->{sha1_b},chg=>'A',file_b=>$m->{file_b}},$deletions);return;
@@ -434,7 +434,7 @@ ($$$$)$self->change_file_prop($fbat,'svn:special',undef);}my$blob=$m->{"sha1_$which"};-return($fh,)if($blob=~/^0{40}$/);+return($fh,)if($blob=~/^0+$/);my$size=$::_repository->cat_blob($blob,$fh);croak"Failed to read object $blob"if($size<0);$fh->flush==0orcroak$!;
@@ -173,7 +173,7 @@ sub delete_entry {# remove entire directories.my($tree)=(command('ls-tree','-z',$self->{c},"./$gpath")-=~/\A040000tree([a-f\d]{40})\t\Q$gpath\E\0/);+=~/\A040000tree($::oid)\t\Q$gpath\E\0/);if($tree){my($ls,$ctx)=command_output_pipe(qw/ls-tree-r--name-only-z/,
@@ -203,7 +203,7 @@ sub open_file {my$gpath=$self->git_path($path);($mode,$blob)=(command('ls-tree','-z',$self->{c},"./$gpath")-=~/\A(\d{6})blob([a-f\d]{40})\t\Q$gpath\E\0/);+=~/\A(\d{6})blob($::oid)\t\Q$gpath\E\0/);unless(defined$mode&&defined$blob){die"$path was not found in commit $self->{c} (r$rev)\n";}
@@ -413,7 +413,7 @@ sub close_file {$hash=$::_repository->hash_and_insert_object(Git::temp_path($fh));-$hash=~/^[a-f\d]{40}$/ordie"not a sha1: $hash\n";+$hash=~/^$::oid$/ordie"not an object ID: $hash\n";Git::temp_release($fb->{base},1);Git::temp_release($fh,1);
@@ -486,11 +486,11 @@ sub gs_fetch_loop_common {$reload_ra->()if$ra_invalid;}# pre-fill the .rev_db since it'll eventually get filled in-# with '0' x40 if something new gets committed+# with '0' x $oid_length if something new gets committedforeachmy$gs(@$gsv){nextif$gs->rev_map_max>=$max;nextifdefined$gs->rev_map_get($max);-$gs->rev_map_set($max,0x40);+$gs->rev_map_set($max,0x$::oid_length);}foreachmy$g(@$globs){my$k="svn-remote.$g->{remote}.$g->{t}-maxRev";
From: brian m. carlson <hidden> Date: 2020-06-19 22:52:12
git-svn has several variables for SHA-1 constants, including short hash
values and full length hash values. Since these are no longer SHA-1
specific, let's start them with "oid" instead of "sha1". Add a
constant, oid_length, which is the length of the hash algorithm in use
in hex. We use the hex version because overwhelmingly that's what's
used by git-svn.
We don't currently set oid_length based on the repository algorithm, but
we will in a future commit.
Signed-off-by: brian m. carlson <redacted>
---
git-svn.perl | 14 ++++++++------
perl/Git/SVN.pm | 10 +++++-----
perl/Git/SVN/Editor.pm | 2 +-
perl/Git/SVN/Log.pm | 2 +-
4 files changed, 15 insertions(+), 13 deletions(-)
@@ -5,7 +5,8 @@usewarnings;usestrict;usevarsqw/$AUTHOR$VERSION-$sha1$sha1_short$_revision$_repository+$oid$oid_short$oid_length+$_revision$_repository$_q$_authors$_authors_prog%users/;$AUTHOR='Eric Wong <normalperson@yhbt.net>';$VERSION='@@GIT_VERSION@@';
@@ -103,8 +104,9 @@ sub _req_svn {}}-$sha1=qr/[a-f\d]{40}/;-$sha1_short=qr/[a-f\d]{4,40}/;+$oid=qr/(?:[a-f\d]{40}(?:[a-f\d]{24})?)/;+$oid_short=qr/[a-f\d]{4,64}/;+$oid_length=40;my($_stdin,$_help,$_edit,$_message,$_file,$_branch_dest,$_template,$_shared,
@@ -582,7 +584,7 @@ sub cmd_set_tree {print"Reading from stdin...\n";@commits=();while(<STDIN>){-if(/\b($sha1_short)\b/o){+if(/\b($oid_short)\b/o){unshift@commits,$1;}}
@@ -1831,7 +1833,7 @@ sub get_tree_from_treeish {if($typeeq'commit'){$expected=(grep/^tree /,command(qw/cat-file commit/,$treeish))[0];-($expected)=($expected=~ /^tree ($sha1)$/o);+($expected)=($expected=~ /^tree ($oid)$/o);die"Unable to get tree from $treeish\n"unless$expected;}elsif($typeeq'tree'){$expected=$treeish;
@@ -2036,7 +2038,7 @@ sub working_head_info {my$hash;my%max;while(<$fh>){-if(m{^commit ($::sha1)$}){+if(m{^commit ($::oid)$}){unshift@$refs,$hashif$hashand$refs;$hash=$1;next;
@@ -1020,7 +1020,7 @@ sub do_git_commit {$tree=$self->tmp_index_do(sub{command_oneline('write-tree')});}-die"Tree is not a valid sha1: $tree\n"if$tree!~/^$::sha1$/o;+die"Tree is not a valid oid $tree\n"if$tree!~/^$::oid$/o;my@exec=('git','commit-tree',$tree);foreach($self->get_commit_parents($log_entry)){
@@ -1048,8 +1048,8 @@ sub do_git_commit {close$out_fhorcroak$!;waitpid$pid,0;croak$?if$?;-if($commit!~/^$::sha1$/o){-die"Failed to commit, invalid sha1: $commit\n";+if($commit!~/^$::oid$/o){+die"Failed to commit, invalid oid: $commit\n";}$self->rev_map_set($log_entry->{revision},$commit,1);
@@ -2150,7 +2150,7 @@ sub rebuild {my$svn_uuid=$self->rewrite_uuid||$self->ra_uuid;my$c;while(<$log>){-if(m{^commit($::sha1)$}){+if(m{^commit($::oid)$}){$c=$1;next;}
From: Eric Wong <hidden> Date: 2020-06-20 00:21:12
"brian m. carlson" [off-list ref] wrote:
The patches are ordered roughly with test-only fixes, then SVN, and then
CVS. There are no additional tests here because everything fails with
SHA-256 before and passes afterwards.
Thanks brian. I took a quick look at this series (including the
CVS stuff) and nothing objectionable jumped out(*).
Consider this series Acked-by: me
(*) except perhaps some of my original code :x
which I hope to fix at some point this year
From: brian m. carlson <hidden> Date: 2020-06-22 18:04:27
I recently sent out part 2/3 of the SHA-256 work and then I realized
that I hadn't built with the CVS and SVN adapters. So this is part
2.5/3 (or, to be pedantic, 3/4).
The patches are ordered roughly with test-only fixes, then SVN, and then
CVS. There are no additional tests here because everything fails with
SHA-256 before and passes afterwards.
Changes from v1:
* Adopt Andreas's suggestion to preserve more of the structure of the
original function.
Range-diff below.
brian m. carlson (14):
t9109: make test hash independent
t9168: make test hash independent
t9108: make test hash independent
t9100: make test work with SHA-256
t9104: make hash size independent
t9101: make hash independent
t/lib-git-svn: make hash size independent
perl: create and switch variables for hash constants
perl: make Git::IndexInfo work with SHA-256
perl: make SVN code hash independent
git-svn: set the OID length based on hash algorithm
git-cvsserver: port to SHA-256
git-cvsimport: port to SHA-256
git-cvsexportcommit: port to SHA-256
git-cvsexportcommit.perl | 14 ++--
git-cvsimport.perl | 8 +--
git-cvsserver.perl | 37 +++++-----
git-svn.perl | 25 ++++---
perl/Git/IndexInfo.pm | 6 +-
perl/Git/SVN.pm | 83 +++++++++++-----------
perl/Git/SVN/Editor.pm | 8 +--
perl/Git/SVN/Fetcher.pm | 6 +-
perl/Git/SVN/Log.pm | 2 +-
perl/Git/SVN/Ra.pm | 4 +-
t/lib-git-svn.sh | 17 +++--
t/t9100-git-svn-basic.sh | 19 ++++-
t/t9101-git-svn-props.sh | 12 ++--
t/t9104-git-svn-follow-parent.sh | 3 +-
t/t9108-git-svn-glob.sh | 4 +-
t/t9109-git-svn-multi-glob.sh | 6 +-
t/t9168-git-svn-partially-globbed-names.sh | 8 +--
17 files changed, 151 insertions(+), 111 deletions(-)
Range-diff against v1:
1: cc8a204394 = 1: cc8a204394 t9109: make test hash independent
2: 814c3423ef = 2: 814c3423ef t9168: make test hash independent
3: 81b1f48671 = 3: 81b1f48671 t9108: make test hash independent
4: 8571f65b17 = 4: 8571f65b17 t9100: make test work with SHA-256
5: d93ded4e74 = 5: d93ded4e74 t9104: make hash size independent
6: 3e401f3e48 = 6: 3e401f3e48 t9101: make hash independent
7: 02cb87e4be ! 7: 17cc278f1d t/lib-git-svn: make hash size independent
@@ t/lib-git-svn.sh: maybe_start_httpd () {
convert_to_rev_db () {
- perl -w -- - "$@" <<\EOF
--use strict;
-+ perl -w -e "use strict; my \$oidlen = $(test_oid rawsz);" -e '
++ perl -w -- - "$(test_oid rawsz)" "$@" <<\EOF
+ use strict;
++my $oidlen = shift;
@ARGV == 2 or die "usage: convert_to_rev_db <input> <output>";
--open my $wr, '+>', $ARGV[1] or die "$!: couldn't open: $ARGV[1]";
--open my $rd, '<', $ARGV[0] or die "$!: couldn't open: $ARGV[0]";
+my $record_size = $oidlen + 4;
+my $hexlen = $oidlen * 2;
-+open my $wr, "+>", $ARGV[1] or die "$!: couldn'\''t open: $ARGV[1]";
-+open my $rd, "<", $ARGV[0] or die "$!: couldn'\''t open: $ARGV[0]";
+ open my $wr, '+>', $ARGV[1] or die "$!: couldn't open: $ARGV[1]";
+ open my $rd, '<', $ARGV[0] or die "$!: couldn't open: $ARGV[0]";
my $size = (stat($rd))[7];
-($size % 24) == 0 or die "Inconsistent size: $size";
-while (sysread($rd, my $buf, 24) == 24) {
@@ t/lib-git-svn.sh: maybe_start_httpd () {
}
}
seek $wr, $offset, 0 or die $!;
- print $wr $c,"\n" or die $!;
- }
- close $wr or die $!;
--close $rd or die $!;
--EOF
-+close $rd or die $!;' \
-+ -- "$@"
- }
-
- require_svnserve () {
8: d59b8ac38e = 8: a59dbf9637 perl: create and switch variables for hash constants
9: d3c991fb52 = 9: c2ff48efa4 perl: make Git::IndexInfo work with SHA-256
10: 7f87880cb2 = 10: 2b5d83ad2f perl: make SVN code hash independent
11: 9100fbd4cf = 11: 66a46baf27 git-svn: set the OID length based on hash algorithm
12: 9655d1f0ea = 12: b2309179f6 git-cvsserver: port to SHA-256
13: 618abf0af1 = 13: 0d5680f2e6 git-cvsimport: port to SHA-256
14: 290f0ec11f = 14: fecbc1fbf9 git-cvsexportcommit: port to SHA-256
From: brian m. carlson <hidden> Date: 2020-06-22 18:04:29
Instead of stripping off the first 41 characters of git log output,
let's just strip off the first space-separated component, which will
work for any size hash.
Signed-off-by: brian m. carlson <redacted>
---
t/t9109-git-svn-multi-glob.sh | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
From: brian m. carlson <hidden> Date: 2020-06-22 18:04:29
Instead of stripping off the first 41 characters of git log output,
let's just strip off the first space-separated component, which will
work for any size hash.
Signed-off-by: brian m. carlson <redacted>
---
t/t9168-git-svn-partially-globbed-names.sh | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
@@ -129,7 +129,7 @@ test_expect_success 'test prefixed globs match just prefix' 'test$(gitrev-parserefs/remotes/three/tags/t_~1)=\$(gitrev-parserefs/remotes/three/branches/b_)&&gitlog--pretty=onelinerefs/remotes/three/tags/t_>actual&&-sed-e"s/^.\{41\}//"actual>output.three&&+cut-d" "-f2-actual>output.three&&test_cmpexpect.threeoutput.three'
@@ -199,7 +199,7 @@ test_expect_success 'test globbing in the middle of the word' 'test$(gitrev-parserefs/remotes/five/tags/fghij~1)=\$(gitrev-parserefs/remotes/five/branches/abcde)&&gitlog--pretty=onelinerefs/remotes/five/tags/fghij>actual&&-sed-e"s/^.\{41\}//"actual>output.five&&+cut-d" "-f2-actual>output.five&&test_cmpexpect.fiveoutput.five'
From: brian m. carlson <hidden> Date: 2020-06-22 18:04:32
git-svn has several variables for SHA-1 constants, including short hash
values and full length hash values. Since these are no longer SHA-1
specific, let's start them with "oid" instead of "sha1". Add a
constant, oid_length, which is the length of the hash algorithm in use
in hex. We use the hex version because overwhelmingly that's what's
used by git-svn.
We don't currently set oid_length based on the repository algorithm, but
we will in a future commit.
Signed-off-by: brian m. carlson <redacted>
---
git-svn.perl | 14 ++++++++------
perl/Git/SVN.pm | 10 +++++-----
perl/Git/SVN/Editor.pm | 2 +-
perl/Git/SVN/Log.pm | 2 +-
4 files changed, 15 insertions(+), 13 deletions(-)
@@ -5,7 +5,8 @@usewarnings;usestrict;usevarsqw/$AUTHOR$VERSION-$sha1$sha1_short$_revision$_repository+$oid$oid_short$oid_length+$_revision$_repository$_q$_authors$_authors_prog%users/;$AUTHOR='Eric Wong <normalperson@yhbt.net>';$VERSION='@@GIT_VERSION@@';
@@ -103,8 +104,9 @@ sub _req_svn {}}-$sha1=qr/[a-f\d]{40}/;-$sha1_short=qr/[a-f\d]{4,40}/;+$oid=qr/(?:[a-f\d]{40}(?:[a-f\d]{24})?)/;+$oid_short=qr/[a-f\d]{4,64}/;+$oid_length=40;my($_stdin,$_help,$_edit,$_message,$_file,$_branch_dest,$_template,$_shared,
@@ -582,7 +584,7 @@ sub cmd_set_tree {print"Reading from stdin...\n";@commits=();while(<STDIN>){-if(/\b($sha1_short)\b/o){+if(/\b($oid_short)\b/o){unshift@commits,$1;}}
@@ -1831,7 +1833,7 @@ sub get_tree_from_treeish {if($typeeq'commit'){$expected=(grep/^tree /,command(qw/cat-file commit/,$treeish))[0];-($expected)=($expected=~ /^tree ($sha1)$/o);+($expected)=($expected=~ /^tree ($oid)$/o);die"Unable to get tree from $treeish\n"unless$expected;}elsif($typeeq'tree'){$expected=$treeish;
@@ -2036,7 +2038,7 @@ sub working_head_info {my$hash;my%max;while(<$fh>){-if(m{^commit ($::sha1)$}){+if(m{^commit ($::oid)$}){unshift@$refs,$hashif$hashand$refs;$hash=$1;next;
@@ -1020,7 +1020,7 @@ sub do_git_commit {$tree=$self->tmp_index_do(sub{command_oneline('write-tree')});}-die"Tree is not a valid sha1: $tree\n"if$tree!~/^$::sha1$/o;+die"Tree is not a valid oid $tree\n"if$tree!~/^$::oid$/o;my@exec=('git','commit-tree',$tree);foreach($self->get_commit_parents($log_entry)){
@@ -1048,8 +1048,8 @@ sub do_git_commit {close$out_fhorcroak$!;waitpid$pid,0;croak$?if$?;-if($commit!~/^$::sha1$/o){-die"Failed to commit, invalid sha1: $commit\n";+if($commit!~/^$::oid$/o){+die"Failed to commit, invalid oid: $commit\n";}$self->rev_map_set($log_entry->{revision},$commit,1);
@@ -2150,7 +2150,7 @@ sub rebuild {my$svn_uuid=$self->rewrite_uuid||$self->ra_uuid;my$c;while(<$log>){-if(m{^commit($::sha1)$}){+if(m{^commit($::oid)$}){$c=$1;next;}
From: brian m. carlson <hidden> Date: 2020-06-22 18:04:33
Instead of stripping off the first 41 characters of git log output,
let's just strip off the first space-separated component, which will
work for any size hash.
Signed-off-by: brian m. carlson <redacted>
---
t/t9108-git-svn-glob.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
From: brian m. carlson <hidden> Date: 2020-06-22 18:04:34
Compute the relevant tree objects for SHA-256 and use those when
appropriate instead of using the SHA-1 ones.
Signed-off-by: brian m. carlson <redacted>
---
t/t9100-git-svn-basic.sh | 19 ++++++++++++++++---
1 file changed, 16 insertions(+), 3 deletions(-)
From: brian m. carlson <hidden> Date: 2020-06-22 18:04:35
Most of the Git modules, git-svn excepted, don't know anything about the
hash algorithm and mostly work. However, when we're printing an
all-zero object ID in Git::IndexInfo, we need to know the hash length.
Since we don't want to change the API to have that information passed
in, let's query the config to find the hash algorithm and compute the
right value.
Signed-off-by: brian m. carlson <redacted>
---
perl/Git/IndexInfo.pm | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
From: brian m. carlson <hidden> Date: 2020-06-22 18:04:36
The record size used in the git svn storage is four bytes plus the
length of the binary hash. Pass the hash length into our Perl
invocation and use it to compute the size of the records.
Signed-off-by: brian m. carlson <redacted>
---
t/lib-git-svn.sh | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
From: brian m. carlson <hidden> Date: 2020-06-22 18:04:38
When we apply a binary patch, we must have the full object ID in the
header in order to apply it; without that, any attempt to apply it will
fail. If we set GIT_DIR to empty, git apply does not know about the
hash algorithm we're using, and consequently any attempt to apply a
patch using SHA-256 will fail, since the object ID is the wrong length.
The reason we set the GIT_DIR environment variable is because we don't
want to modify the index; we just want to know whether the patch
applies. Instead, let's just use a temporary file for the index, which
will be cleaned up automatically when the object goes out of scope.
Additionally, read the configuration for the repository and compute the
length of an object ID based on it. Use that when matching object IDs
with a regex or computing the all-zeros object ID.
Signed-off-by: brian m. carlson <redacted>
---
git-cvsexportcommit.perl | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
@@ -22,6 +22,10 @@my$repo=Git->repository();$opt_w=$repo->config('cvsexportcommit.cvsdir')unlessdefined$opt_w;+my$tmpdir=File::Temp->newdir;+my$hash_algo=$repo->config('extensions.objectformat')||'sha1';+my$hexsz=$hash_algoeq'sha256'?64:40;+if($opt_w||$opt_W){# Remember where GIT_DIR is before changing to CVS checkoutunless($ENV{GIT_DIR}){
@@ -96,7 +100,7 @@}if($stageeq'headers'){-if($line=~m/^parent (\w{40})$/){# found a parent+if($line=~m/^parent ([0-9a-f]{$hexsz})$/){# found a parentpush@parents,$1;}elsif($line=~m/^author (.+) \d+ [-+]\d+$/){$author=$1;
@@ -111,7 +115,7 @@}}-my$noparent="0000000000000000000000000000000000000000";+my$noparent="0"x$hexsz;if($parent){my$found;# double check that it's a valid parent
@@ -174,7 +178,7 @@print"Checking if patch will apply\n";my@stat;-openAPPLY,"GIT_DIR= git-apply $context --summary --numstat<.cvsexportcommit.diff|"||die"cannot patch";+openAPPLY,"GIT_INDEX_FILE=$tmpdir/index git-apply $context --summary --numstat<.cvsexportcommit.diff|"||die"cannot patch";@stat=<APPLY>;closeAPPLY||die"Cannot patch";my(@bfiles,@files,@afiles,@dfiles);
@@ -407,7 +411,7 @@if($opt_W){system("git checkout $go_back_to")&&die"cannot move back to $go_back_to";-if(!($go_back_to=~ /^[0-9a-fA-F]{40}$/)){+if(!($go_back_to=~ /^[0-9a-fA-F]{$hexsz}$/)){system("git symbolic-ref HEAD $go_back_to")&&die"cannot move back to $go_back_to";}
From: brian m. carlson <hidden> Date: 2020-06-22 18:04:39
When reading the configuration or when creating a new repository, load
the extensions.objectFormat value and set the object ID length to 64 if
it's "sha256". Note that we use the hex length in git-svn because most
of our processing is done on hex values, not binary ones.
Signed-off-by: brian m. carlson <redacted>
---
git-svn.perl | 7 +++++++
1 file changed, 7 insertions(+)
From: brian m. carlson <hidden> Date: 2020-06-22 18:04:42
Instead of calling the function is_sha1, call it is_oid and update it to
match either a SHA-1 or a SHA-256 hex object ID.
Signed-off-by: brian m. carlson <redacted>
---
git-cvsimport.perl | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
@@ -810,7 +810,7 @@ ()open(my$fh,'-|',qw(git write-tree))ordie"unable to open git write-tree: $!";chomp(my$tree=<$fh>);-is_sha1($tree)+is_oid($tree)ordie"Cannot get tree id ($tree): $!";close($fh)ordie"Error running git write-tree: $?\n";
@@ -896,7 +896,7 @@ sub commit {print"Committed patch $patchset ($branch $commit_date)\n"if$opt_v;chomp(my$cid=<$commit_read>);-is_sha1($cid)ordie"Cannot get commit id ($cid): $!\n";+is_oid($cid)ordie"Cannot get commit id ($cid): $!\n";print"Commit ID $cid\n"if$opt_v;close($commit_read);
From: brian m. carlson <hidden> Date: 2020-06-22 18:04:43
The size of a record in the database used by git svn is four bytes plus
the length of the binary hash. Instead of hard-coding 24, compute this
value based on the size of the hash in use.
Signed-off-by: brian m. carlson <redacted>
---
t/t9104-git-svn-follow-parent.sh | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
@@ -161,6 +161,7 @@ test_expect_success "track initial change if it was only made to parent" '' test_expect_success"follow-parent is atomic"'+record_size=$(($(test_oidrawsz)+4))&&(cdwc&&svn_cmdup&&
@@ -186,7 +187,7 @@ test_expect_success "follow-parent is atomic" 'mkdir-p"$GIT_DIR"/svn/refs/remotes/flunk@18&&rev_map=$(cd"$GIT_DIR"/svn/refs/remotes/stunk&&ls.rev_map*)&&ddif="$GIT_DIR"/svn/refs/remotes/stunk/$rev_map\-of="$GIT_DIR"/svn/refs/remotes/flunk@18/$rev_mapbs=24count=1&&+of="$GIT_DIR"/svn/refs/remotes/flunk@18/$rev_mapbs=$record_sizecount=1&&rm-rf"$GIT_DIR"/svn/refs/remotes/stunk&&gitsvninit--minimize-url-iflunk"$svnrepo"/flunk&&gitsvnfetch-iflunk&&
From: brian m. carlson <hidden> Date: 2020-06-22 18:04:44
The code of git-cvsserver currently has several hard-coded 20 and 40
constants that are the length of SHA-1. When parsing the configuration
file, read the extensions.objectformat configuration setting as well as
CVS-related ones and adjust the hash sizes accordingly. Use these
computed values in all the places we match object IDs.
Signed-off-by: brian m. carlson <redacted>
---
git-cvsserver.perl | 37 ++++++++++++++++++++-----------------
1 file changed, 20 insertions(+), 17 deletions(-)
@@ -2375,7 +2378,7 @@ sub req_annotateprint"E ***************\n";while(<ANNOTATE>){-if(m/^([a-zA-Z0-9]{40})\t\([^\)]*\)(.*)$/i)+if(m/^([a-zA-Z0-9]{$state->{hexsz}})\t\([^\)]*\)(.*)$/i){my$commithash=$1;my$data=$2;
@@ -2852,7 +2855,7 @@ sub transmitfilereturn;}-die"Need filehash"unless(defined($filehash)and$filehash=~ /^[a-zA-Z0-9]{40}$/);+die"Need filehash"unless(defined($filehash)and$filehash=~ /^[a-zA-Z0-9]{$state->{hexsz}}$/);my$type=safe_pipe_capture('git','cat-file','-t',$filehash);chomp$type;
@@ -3042,7 +3045,7 @@ sub ensureWorkTreemy$ver=safe_pipe_capture('git','show-ref','-s',"refs/heads/$state->{module}");chomp$ver;-if($ver!~/^[0-9a-f]{40}$/)+if($ver!~/^[0-9a-f]{$state->{hexsz}}$/){$log->warn("Error from git show-ref -s refs/head$state->{module}");print"error 1 cannot find the current HEAD of module";
@@ -3281,7 +3284,7 @@ sub open_blob_or_die}elsif($srcTypeeq"sha1"){-unless(defined($name)and$name=~ /^[a-zA-Z0-9]{40}$/)+unless(defined($name)and$name=~ /^[a-zA-Z0-9]{$state->{hexsz}}$/){$log->warn("Need filehash");die"Need filehash\n";
@@ -3817,7 +3820,7 @@ sub updatechomp$commitsha1;my$commitinfo=::safe_pipe_capture('git','cat-file','commit',$self->{module});-unless($commitinfo=~ /tree\s+[a-zA-Z0-9]{40}/)+unless($commitinfo=~ /tree\s+[a-zA-Z0-9]{$state->{hexsz}}/){die("Invalid module '$self->{module}'");}
@@ -3957,7 +3960,7 @@ sub updatewhile(<FILELIST>){chomp;-unless( /^:\d{6}\s+([0-7]{6})\s+[a-f0-9]{40}\s+([a-f0-9]{40})\s+(\w)$/o)+unless( /^:\d{6}\s+([0-7]{6})\s+[a-f0-9]{$state->{hexsz}}\s+([a-f0-9]{$state->{hexsz}})\s+(\w)$/o){die("Couldn't process git-diff-tree line : $_");}
@@ -4625,11 +4628,11 @@ sub getmeta$db_query->execute($filename,$intRev);$meta=$db_query->fetchrow_hashref;}-elsif($revision=~ /^2\.1\.1\.2000(\.[1-3][0-9][0-9]){20}$/)+elsif($revision=~ /^2\.1\.1\.2000(\.[1-3][0-9][0-9]){$state->{rawsz}}$/){my($commitHash)=($revision=~/^2\.1\.1\.2000(.*)$/);$commitHash=~s/\.([0-9]+)/sprintf("%02x",$1-100)/eg;-if($commitHash=~/^[0-9a-f]{40}$/)+if($commitHash=~/^[0-9a-f]{$state->{hexsz}}$/){return$self->getMetaFromCommithash($filename,$commitHash);}
@@ -4639,7 +4642,7 @@ sub getmeta$log->warning("failed get $revision with commithash=$commitHash");undef$revision;}-elsif($revision=~ /^[0-9a-f]{40}$/)+elsif($revision=~ /^[0-9a-f]{$state->{hexsz}}$/){# Try DB first. This is mostly only useful for req_annotate(),# which only calls this for stuff that should already be in
@@ -4658,7 +4661,7 @@ sub getmetaif(!$meta){my($revCommit)=$self->lookupCommitRef($revision);-if($revCommit=~/^[0-9a-f]{40}$/)+if($revCommit=~/^[0-9a-f]{$state->{hexsz}}$/){return$self->getMetaFromCommithash($filename,$revCommit);}
@@ -4672,7 +4675,7 @@ sub getmetaelse{my($revCommit)=$self->lookupCommitRef($revision);-if($revCommit=~/^[0-9a-f]{40}$/)+if($revCommit=~/^[0-9a-f]{$state->{hexsz}}$/){return$self->getMetaFromCommithash($filename,$revCommit);}
@@ -4863,7 +4866,7 @@ sub lookupCommitRef$commitHash=::safe_pipe_capture("git","rev-parse","--verify","--quiet",$self->unescapeRefName($ref));$commitHash=~s/\s*$//;-if(!($commitHash=~/^[0-9a-f]{40}$/))+if(!($commitHash=~/^[0-9a-f]{$state->{hexsz}}$/)){$commitHash=undef;}
@@ -4909,7 +4912,7 @@ sub commitmessagemy$commithash=shift;my$tablename=$self->tablename("commitmsgs");-die("Need commithash")unless(defined($commithash)and$commithash=~ /^[a-zA-Z0-9]{40}$/);+die("Need commithash")unless(defined($commithash)and$commithash=~ /^[a-zA-Z0-9]{$state->{hexsz}}$/);my$db_query;$db_query=$self->{dbh}->prepare_cached("SELECT value FROM $tablename WHERE key=?",{},1);
From: brian m. carlson <hidden> Date: 2020-06-22 18:05:00
Instead of hard-coding the object ID for our test .gitignore file, let's
compute it.
Signed-off-by: brian m. carlson <redacted>
---
t/t9101-git-svn-props.sh | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
From: brian m. carlson <hidden> Date: 2020-06-22 18:05:03
There are several places throughout git-svn that use various hard-coded
constants. For matching object IDs, use the $oid variable. Compute the
record size we use for our revision storage based on the object ID.
When parsing the revision map format, use a wildcard in the pack format
since we know that the data we're parsing is always exactly the record
size. This lets us continue to use a constant for the pack format.
Finally, update several comments to reflect the fact that an object ID
may be of one of multiple sizes.
Signed-off-by: brian m. carlson <redacted>
---
git-svn.perl | 4 +--
perl/Git/SVN.pm | 73 +++++++++++++++++++++--------------------
perl/Git/SVN/Editor.pm | 6 ++--
perl/Git/SVN/Fetcher.pm | 6 ++--
perl/Git/SVN/Ra.pm | 4 +--
5 files changed, 48 insertions(+), 45 deletions(-)
@@ -2087,10 +2087,10 @@ sub rebuild_from_rev_db {openmy$fh,'<',$pathorcroak"open: $!";binmode$fhorcroak"binmode: $!";while(<$fh>){-length($_)==41orcroak"inconsistent size in ($_) != 41";+length($_)==$::oid_length+1orcroak"inconsistent size in ($_)";chomp($_);++$r;-nextif$_eq('0'x40);+nextif$_eq('0'x$::oid_length);$self->rev_map_set($r,$_);print"r$r = $_\n";}
@@ -2196,9 +2196,9 @@ sub rebuild {# (mainly tags)## The format is this:-# - 24 bytes for every record,+# - 24 or 36 bytes for every record,# * 4 bytes for the integer representing an SVN revision number-# * 20 bytes representing the sha1 of a git commit+# * 20 or 32 bytes representing the oid of a git commit# - No empty padding records like the old format# (except the last record, which can be overwritten)# - new records are written append-only since SVN revision numbers
@@ -2207,7 +2207,7 @@ sub rebuild {# - Piping the file to xxd -c24 is a good way of dumping it for# viewing or editing (piped back through xxd -r), should the need# ever arise.-# - The last record can be padding revision with an all-zero sha1+# - The last record can be padding revision with an all-zero oid# This is used to optimize fetch performance when using multiple# "fetch" directives in .git/config#
@@ -2215,38 +2215,39 @@ sub rebuild {sub_rev_map_set{my($fh,$rev,$commit)=@_;+my$record_size=($::oid_length/2)+4;binmode$fhorcroak"binmode: $!";my$size=(stat($fh))[7];-($size%24)==0orcroak"inconsistent size: $size";+($size%$record_size)==0orcroak"inconsistent size: $size";my$wr_offset=0;if($size>0){-sysseek($fh,-24,SEEK_END)orcroak"seek: $!";-my$read=sysread($fh,my$buf,24)orcroak"read: $!";-$read==24orcroak"read only $read bytes (!= 24)";+sysseek($fh,-$record_size,SEEK_END)orcroak"seek: $!";+my$read=sysread($fh,my$buf,$record_size)orcroak"read: $!";+$read==$record_sizeorcroak"read only $read bytes (!= $record_size)";my($last_rev,$last_commit)=unpack(rev_map_fmt,$buf);-if($last_commiteq('0'x40)){-if($size>=48){-sysseek($fh,-48,SEEK_END)orcroak"seek: $!";-$read=sysread($fh,$buf,24)or+if($last_commiteq('0'x$::oid_length)){+if($size>=($record_size*2)){+sysseek($fh,-($record_size*2),SEEK_END)orcroak"seek: $!";+$read=sysread($fh,$buf,$record_size)orcroak"read: $!";-$read==24or-croak"read only $read bytes (!= 24)";+$read==$record_sizeor+croak"read only $read bytes (!= $record_size)";($last_rev,$last_commit)=unpack(rev_map_fmt,$buf);-if($last_commiteq('0'x40)){+if($last_commiteq('0'x$::oid_length)){croak"inconsistent .rev_map\n";}}if($last_rev>=$rev){croak"last_rev is higher!: $last_rev >= $rev";}-$wr_offset=-24;+$wr_offset=-$record_size;}}sysseek($fh,$wr_offset,SEEK_END)orcroak"seek: $!";-syswrite($fh,pack(rev_map_fmt,$rev,$commit),24)==24or+syswrite($fh,pack(rev_map_fmt,$rev,$commit),$record_size)==$record_sizeorcroak"write: $!";}
@@ -2271,7 +2272,7 @@ sub mkfile {subrev_map_set{my($self,$rev,$commit,$update_ref,$uuid)=@_;defined$commitordie"missing arg3\n";-length$commit==40ordie"arg3 must be a full SHA1 hexsum\n";+$commit=~/^$::oid$/ordie"arg3 must be a full hex object ID\n";my$db=$self->map_path($uuid);my$db_lock="$db.lock";my$sigmask;
@@ -2344,29 +2345,30 @@ sub rev_map_max {subrev_map_max_norebuild{my($self,$want_commit)=@_;+my$record_size=($::oid_length/2)+4;my$map_path=$self->map_path;stat$map_pathorreturn$want_commit?(0,undef):0;sysopen(my$fh,$map_path,O_RDONLY)orcroak"open: $!";binmode$fhorcroak"binmode: $!";my$size=(stat($fh))[7];-($size%24)==0orcroak"inconsistent size: $size";+($size%$record_size)==0orcroak"inconsistent size: $size";if($size==0){close$fhorcroak"close: $!";return$want_commit?(0,undef):0;}-sysseek($fh,-24,SEEK_END)orcroak"seek: $!";-sysread($fh,my$buf,24)==24orcroak"read: $!";+sysseek($fh,-$record_size,SEEK_END)orcroak"seek: $!";+sysread($fh,my$buf,$record_size)==$record_sizeorcroak"read: $!";my($r,$c)=unpack(rev_map_fmt,$buf);-if($want_commit&&$ceq('0'x40)){-if($size<48){+if($want_commit&&$ceq('0'x$::oid_length)){+if($size<$record_size*2){return$want_commit?(0,undef):0;}-sysseek($fh,-48,SEEK_END)orcroak"seek: $!";-sysread($fh,$buf,24)==24orcroak"read: $!";+sysseek($fh,-($record_size*2),SEEK_END)orcroak"seek: $!";+sysread($fh,$buf,$record_size)==$record_sizeorcroak"read: $!";($r,$c)=unpack(rev_map_fmt,$buf);-if($ceq('0'x40)){+if($ceq('0'x$::oid_length)){croak"Penultimate record is all-zeroes in $map_path";}}
@@ -400,12 +400,12 @@ sub T {($m->{mode_b}!~/^120/&&$m->{mode_a}=~/^120/)){$self->D({mode_a=>$m->{mode_a},mode_b=>'000000',-sha1_a=>$m->{sha1_a},sha1_b=>'0'x40,+sha1_a=>$m->{sha1_a},sha1_b=>'0'x$::oid_length,chg=>'D',file_b=>$m->{file_b}},$deletions);$self->A({mode_a=>'000000',mode_b=>$m->{mode_b},-sha1_a=>'0'x40,sha1_b=>$m->{sha1_b},+sha1_a=>'0'x$::oid_length,sha1_b=>$m->{sha1_b},chg=>'A',file_b=>$m->{file_b}},$deletions);return;
@@ -434,7 +434,7 @@ ($$$$)$self->change_file_prop($fbat,'svn:special',undef);}my$blob=$m->{"sha1_$which"};-return($fh,)if($blob=~/^0{40}$/);+return($fh,)if($blob=~/^0+$/);my$size=$::_repository->cat_blob($blob,$fh);croak"Failed to read object $blob"if($size<0);$fh->flush==0orcroak$!;
@@ -173,7 +173,7 @@ sub delete_entry {# remove entire directories.my($tree)=(command('ls-tree','-z',$self->{c},"./$gpath")-=~/\A040000tree([a-f\d]{40})\t\Q$gpath\E\0/);+=~/\A040000tree($::oid)\t\Q$gpath\E\0/);if($tree){my($ls,$ctx)=command_output_pipe(qw/ls-tree-r--name-only-z/,
@@ -203,7 +203,7 @@ sub open_file {my$gpath=$self->git_path($path);($mode,$blob)=(command('ls-tree','-z',$self->{c},"./$gpath")-=~/\A(\d{6})blob([a-f\d]{40})\t\Q$gpath\E\0/);+=~/\A(\d{6})blob($::oid)\t\Q$gpath\E\0/);unless(defined$mode&&defined$blob){die"$path was not found in commit $self->{c} (r$rev)\n";}
@@ -413,7 +413,7 @@ sub close_file {$hash=$::_repository->hash_and_insert_object(Git::temp_path($fh));-$hash=~/^[a-f\d]{40}$/ordie"not a sha1: $hash\n";+$hash=~/^$::oid$/ordie"not an object ID: $hash\n";Git::temp_release($fb->{base},1);Git::temp_release($fh,1);
@@ -486,11 +486,11 @@ sub gs_fetch_loop_common {$reload_ra->()if$ra_invalid;}# pre-fill the .rev_db since it'll eventually get filled in-# with '0' x40 if something new gets committed+# with '0' x $oid_length if something new gets committedforeachmy$gs(@$gsv){nextif$gs->rev_map_max>=$max;nextifdefined$gs->rev_map_get($max);-$gs->rev_map_set($max,0x40);+$gs->rev_map_set($max,0x$::oid_length);}foreachmy$g(@$globs){my$k="svn-remote.$g->{remote}.$g->{t}-maxRev";