Re: Figured out how to get Mozilla into git

"Jon Smirl" [off-list ref] writes:

Here's a new transport problem. When using git-clone to fetch Martin's
tree it kept failing for me at dreamhost. I had a parallel fetch
running on my local machine which has a much slower net connection. It
finally finished and I am watching the end phase where it prints all
of the 'walk' messages. The git-http-fetch process has jumped up to
800MB in size after being 2MB during the download. dreamhost has a
500MB process size limit so that is why my fetches kept failing there.

The http-fetch process uses by mmaping the downloaded pack, and
if I recall correctly we are talking about 600MB pack, so 500MB
limit sounds impossible, perhaps?

On 6/10/06, Junio C Hamano [off-list ref] wrote:

"Jon Smirl" [off-list ref] writes:

quoted

Here's a new transport problem. When using git-clone to fetch Martin's
tree it kept failing for me at dreamhost. I had a parallel fetch
running on my local machine which has a much slower net connection. It
finally finished and I am watching the end phase where it prints all
of the 'walk' messages. The git-http-fetch process has jumped up to
800MB in size after being 2MB during the download. dreamhost has a
500MB process size limit so that is why my fetches kept failing there.

The http-fetch process uses by mmaping the downloaded pack, and
if I recall correctly we are talking about 600MB pack, so 500MB
limit sounds impossible, perhaps?

The fetch on my local machine failed too. It left nothing behind, now
I have to download the 680MB again.

walk 1f19465388a4ef7aff7527a13f16122a809487d4
walk c3ca840256e3767d08c649f8d2761a1a887351ab
walk 7a74e42699320c02b814b88beadb1ae65009e745
error: Couldn't get
http://mirrors.catalyst.net.nz/pub/mozilla.git//refs/tags/JS%5F1%5F7%5FALPHA%5FBASE
for tags/JS_1_7_ALPHA_BASE
Couldn't resolve host 'mirrors.catalyst.net.nz'
error: Could not interpret tags/JS_1_7_ALPHA_BASE as something to pull
[jonsmirl@jonsmirl mozgit]$ cg update
There is no GIT repository here (.git not found)
[jonsmirl@jonsmirl mozgit]$ ls -a
.  ..
[jonsmirl@jonsmirl mozgit]$




-- 
Jon Smirl
jonsmirl@gmail.com

"Jon Smirl" [off-list ref] wrote:

On 6/10/06, Junio C Hamano [off-list ref] wrote:

quoted

"Jon Smirl" [off-list ref] writes:

quoted

Here's a new transport problem. When using git-clone to fetch Martin's
tree it kept failing for me at dreamhost. I had a parallel fetch
running on my local machine which has a much slower net connection. It
finally finished and I am watching the end phase where it prints all
of the 'walk' messages. The git-http-fetch process has jumped up to
800MB in size after being 2MB during the download. dreamhost has a
500MB process size limit so that is why my fetches kept failing there.

The http-fetch process uses by mmaping the downloaded pack, and
if I recall correctly we are talking about 600MB pack, so 500MB
limit sounds impossible, perhaps?

The fetch on my local machine failed too. It left nothing behind, now
I have to download the 680MB again.

That's sad.  Could git-clone be changed to not remove .git directory if
fetching objects fails (after other files in the .git directory have
been fetched)?  You could then hopefully continue with git-pull.

-- 
http://onion.dynserv.net/~timo/

Dear diary, on Sat, Jun 10, 2006 at 05:44:58PM CEST, I got a letter
where Jon Smirl [off-list ref] said that...

The fetch on my local machine failed too. It left nothing behind, now
I have to download the 680MB again.

walk 1f19465388a4ef7aff7527a13f16122a809487d4
walk c3ca840256e3767d08c649f8d2761a1a887351ab
walk 7a74e42699320c02b814b88beadb1ae65009e745
error: Couldn't get
http://mirrors.catalyst.net.nz/pub/mozilla.git//refs/tags/JS%5F1%5F7%5FALPHA%5FBASE
for tags/JS_1_7_ALPHA_BASE
Couldn't resolve host 'mirrors.catalyst.net.nz'
error: Could not interpret tags/JS_1_7_ALPHA_BASE as something to pull
[jonsmirl@jonsmirl mozgit]$ cg update
There is no GIT repository here (.git not found)
[jonsmirl@jonsmirl mozgit]$ ls -a
.  ..
[jonsmirl@jonsmirl mozgit]$

  You could try with cg-clone, which won't delete the repository if
things fail. It will clone only the master branch, though.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
A person is just about as big as the things that make them angry.

On (10/06/06 11:44), Jon Smirl wrote:

Date:	Sat, 10 Jun 2006 11:44:58 -0400
From:	"Jon Smirl" <redacted>
To:	"Junio C Hamano" <redacted>
Subject: Re: Figured out how to get Mozilla into git
Cc:	git@vger.kernel.org

On 6/10/06, Junio C Hamano [off-list ref] wrote:

quoted

"Jon Smirl" [off-list ref] writes:

quoted

Here's a new transport problem. When using git-clone to fetch Martin's
tree it kept failing for me at dreamhost. I had a parallel fetch
running on my local machine which has a much slower net connection. It
finally finished and I am watching the end phase where it prints all
of the 'walk' messages. The git-http-fetch process has jumped up to
800MB in size after being 2MB during the download. dreamhost has a
500MB process size limit so that is why my fetches kept failing there.

The http-fetch process uses by mmaping the downloaded pack, and
if I recall correctly we are talking about 600MB pack, so 500MB
limit sounds impossible, perhaps?

The fetch on my local machine failed too. It left nothing behind, now
I have to download the 680MB again.

walk 1f19465388a4ef7aff7527a13f16122a809487d4
walk c3ca840256e3767d08c649f8d2761a1a887351ab
walk 7a74e42699320c02b814b88beadb1ae65009e745
error: Couldn't get
http://mirrors.catalyst.net.nz/pub/mozilla.git//refs/tags/JS%5F1%5F7%5FALPHA%5FBASE
for tags/JS_1_7_ALPHA_BASE
Couldn't resolve host 'mirrors.catalyst.net.nz'
error: Could not interpret tags/JS_1_7_ALPHA_BASE as something to pull
[jonsmirl@jonsmirl mozgit]$ cg update
There is no GIT repository here (.git not found)
[jonsmirl@jonsmirl mozgit]$ ls -a
.  ..
[jonsmirl@jonsmirl mozgit]$

To prevent repeat (on this repo) your could grab it with a browser:
-mkdir tmp; cd tmp; git init-db;
-copy  mirror../pu/mozilla.git/objects/*  to .git/objects/
-copy   --||---.git/info/refs to refsinfo in tmp-dir
gawk '{if  ($2 !~ /\^\{\}$/) print $1 > sprintf(".git/%s",$2);}' refsinfo
 to extract branches and tags into ./git/refs/{heads,tags}
start playing (after a backup) with git-fsck-objects, git-checkout etc.
 
-- 
Lars Johannsen 
mail@Lars-johannsen.dk