Bug?: import-tars misbehaves on Subversion tarballs

import-tars behaves very oddly when I try to import the Subversion
tarballs. (For example,
http://subversion.tigris.org/downloads/subversion-1.4.3.tar.bz2
triggers this problem.) It creates two toplevel directories,
subversion-1.4.3 and subversion-1.4.3subversion. The former seems to
contain at least almost all files; the latter has only a handful of
files, all with very long names.

When I unpack it with GNU tar, I get all the files under a single
"subversion-1.4.3" directory, as expected.

Could it simply be that import-tars can't handle long filenames
somehow? (This is pure speculation, since I know absolutely nothing
about the tar format.)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Hello Karl,

Karl Hasselström wrote:

import-tars behaves very oddly when I try to import the Subversion
tarballs. (For example,
http://subversion.tigris.org/downloads/subversion-1.4.3.tar.bz2
triggers this problem.) It creates two toplevel directories,
subversion-1.4.3 and subversion-1.4.3subversion. The former seems to
contain at least almost all files; the latter has only a handful of
files, all with very long names.

When I unpack it with GNU tar, I get all the files under a single
"subversion-1.4.3" directory, as expected.

Could it simply be that import-tars can't handle long filenames
somehow? (This is pure speculation, since I know absolutely nothing
about the tar format.)

I don't know much, but there are two locations that make up the name
(i.e. prefix + name).  Can you try the following patch:

---
 contrib/fast-import/import-tars.perl |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/contrib/fast-import/import-tars.perl b/contrib/fast-import/import-tars.perl
index 5585a8b..5f3f742 100755
--- a/contrib/fast-import/import-tars.perl
+++ b/contrib/fast-import/import-tars.perl

@@ -64,7 +64,7 @@ foreach my $tar_file (@ARGV)
 		}
 		print FI "\n";
 
-		my $path = "$prefix$name";
+		my $path = "$prefix/$name";
 		$files{$path} = [$next_mark++, $mode];
 
 		$commit_time = $mtime if $mtime > $commit_time;

-- 
1.5.1.1.190.g74474

If it works, we still need a commit log ...

Best regards
Uwe

-- 
Uwe Kleine-König

http://www.google.com/search?q=5%2B7

Hi,

Am 24.04.2007 um 09:39 schrieb Karl Hasselström:

import-tars behaves very oddly when I try to import the Subversion
tarballs. (For example,
http://subversion.tigris.org/downloads/subversion-1.4.3.tar.bz2
triggers this problem.) It creates two toplevel directories,
subversion-1.4.3 and subversion-1.4.3subversion. The former seems to
contain at least almost all files; the latter has only a handful of
files, all with very long names.

I looked at this tarball with midnight commander (under MacOS X) and  
i saw 2 directories, subversion/ and subversion-1.4.3/. Then i looked  
at it with GNU tar (1.14) and saw only subversion-1.4.3/.

Could it simply be that import-tars can't handle long filenames
somehow? (This is pure speculation, since I know absolutely nothing
about the tar format.)

I don't know, but mc does the same thing. I'll look at it later (i  
have more time at 18:00 +0200).


bye

On 2007-04-24 10:40:37 +0200, Uwe Kleine-König wrote:

I don't know much, but there are two locations that make up the name
(i.e. prefix + name). Can you try the following patch:

No, it still breaks, but in a new and interesting way: Now I _only_
get the files with long pathnames!

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

add a / between the prefix and name fields of the tar archive if prefix
is non-empty.

Signed-off-by: Uwe Kleine-König <redacted>
---

No, it still breaks, but in a new and interesting way: Now I _only_
get the files with long pathnames!

I don't know exactly how standard-conformant it is not to include a
trailing / after prefix, but the subversion tar does it. 

I checked for a description of the tar format, the best thing I could
find is[1]:

	On USTAR format archives, the value of the prefix field, if
	non-null, is prefixed to the name field to allow names longer
	then 100 characters

But it's not specifying if an additional / is needed.


With this patch I get all the filenames right.

Now the common prefix subversion-1.4.3 is stripped, but probably that's
a feature.

BTW I shortly tested a tar that had an entry with prefix ending in / and
fast-import did the right thing.  So something more complicated as:

	if ($prefix and $prefix[-1] eq "/") ...

should not be needed.

Best regards
Uwe

[1] http://www.mkssoftware.com/docs/man4/tar.4.asp

 contrib/fast-import/import-tars.perl |    7 ++++++-
 1 files changed, 6 insertions(+), 1 deletions(-)

diff --git a/contrib/fast-import/import-tars.perl b/contrib/fast-import/import-tars.perl
index 5585a8b..1842146 100755
--- a/contrib/fast-import/import-tars.perl
+++ b/contrib/fast-import/import-tars.perl

@@ -64,7 +64,12 @@ foreach my $tar_file (@ARGV)
 		}
 		print FI "\n";
 
-		my $path = "$prefix$name";
+		my $path;
+		if ($prefix) {
+			$path = "$prefix/$name";
+		} else {
+			$path = "$name";
+		}
 		$files{$path} = [$next_mark++, $mode];
 
 		$commit_time = $mtime if $mtime > $commit_time;

-- 
1.5.2.rc0.16.g0f57d

-- 
Uwe Kleine-König

exit vi, lesson IV:
Z Z

NB: may write current file

On 2007-04-24 13:51:04 +0200, Uwe Kleine-König wrote:

add a / between the prefix and name fields of the tar archive if
prefix is non-empty.

Signed-off-by: Uwe Kleine-König <redacted>

Acked-by: Karl Hasselström <redacted>

This solves my problem with the Subversion tarballs. Thanks!

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

On 2007-04-24 13:51:04 +0200, Uwe Kleine-König wrote:

Now the common prefix subversion-1.4.3 is stripped, but probably
that's a feature.

Yes, there's code in import-tars that strips the topmost directory if
and only if all paths in the tar are contained in it.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Uwe Kleine-K??nig [off-list ref] wrote:

add a / between the prefix and name fields of the tar archive if prefix
is non-empty.

Thanks, this is in my fastimport tree now.

-- 
Shawn.