fatal: Out of memory, malloc failed

Hello,

Its not a huge push Im trying to do here (<about 150Mb) but always malloc fails!

$ git push upload master
Fetching remote heads...
  refs/
  refs/heads/
  refs/tags/
updating 'refs/heads/master'
  from 0000000000000000000000000000000000000000
  to   4dc5525d0cc8ca24883c3fa1d5c29b3a105b4610
    sending 1024 objects
fatal: Out of memory, malloc failed

Any ideas? Thanks.

On Fri, 13 Apr 2007, Alan Larkin wrote:

Its not a huge push Im trying to do here (<about 150Mb) but always malloc fails!

Any huge objects?

Also, it might be interesting to run it under gdb, and put a breakpoint on 
the "die" function, so that it stops where it runs out of memory. Then. at 
that point, you can:

 - do a "where" in gdb to see what allocation it is (and ask it how big 
   it was by printing out the value of "size").

   It may be something totally uninteresting (just some random object that 
   happened to push things over the limit), but statistically, malloc 
   failures tend to happen to big objects, and sometimes just because 
   somebody needed a huge area that won't fit in the virtual address 
   space.

 - check with "ps" what the size of the process is. Maybe you even just 
   have some process limit set that causes brk/mmap to return failure 
   earlier than necessary..

   (It can also be interesting to look at /proc/<pid>/maps, in case
   there are big mmaps that fill up the VM etc)

Sometimes it's also a good idea to have a swap file.  You may not even
*need* to actually page, but it gives thew VM layer much more freedom,
especially if your distro has set the flags to disable memory "overcommit".

		Linus

Linus Torvalds wrote:

On Fri, 13 Apr 2007, Alan Larkin wrote:

quoted

Its not a huge push Im trying to do here (<about 150Mb) but always malloc fails!

Any huge objects?

Also, it might be interesting to run it under gdb, and put a breakpoint on 
the "die" function, so that it stops where it runs out of memory. Then. at 
that point, you can:

 - do a "where" in gdb to see what allocation it is (and ask it how big 
   it was by printing out the value of "size").

   It may be something totally uninteresting (just some random object that 
   happened to push things over the limit), but statistically, malloc 
   failures tend to happen to big objects, and sometimes just because 
   somebody needed a huge area that won't fit in the virtual address 
   space.

 - check with "ps" what the size of the process is. Maybe you even just 
   have some process limit set that causes brk/mmap to return failure 
   earlier than necessary..

   (It can also be interesting to look at /proc/<pid>/maps, in case
   there are big mmaps that fill up the VM etc)

Sometimes it's also a good idea to have a swap file.  You may not even
*need* to actually page, but it gives thew VM layer much more freedom,
especially if your distro has set the flags to disable memory "overcommit".

		Linus

There were a couple of big files. I removed a 72Mb one (making 47Mb the biggest one left in the
project) and made the push and it worked. I later pulled the project down to a different machine,
added the 72Mb file back in, and pushed to the server and it worked. So apparently it's a platform
specific problem. If anybody's particularly interested I could replicate it under gdb and pass on
any info, but if not I wont ... job's done, Im happy.

Alan Larkin, Sun, Apr 15, 2007 21:06:53 +0200:

quoted

quoted

Its not a huge push Im trying to do here (<about 150Mb) but
always malloc fails!

Any huge objects?

There were a couple of big files. I removed a 72Mb one (making 47Mb
the biggest one left in the project) and made the push and it
worked. I later pulled the project down to a different machine,
added the 72Mb file back in, and pushed to the server and it worked.
So apparently it's a platform specific problem. If anybody's
particularly interested I could replicate it under gdb and pass on
any info, but if not I wont ... job's done, Im happy.

72Mb is nothing. Is it Windows/cygwin, by any chance? (I see the
problem there sometimes. Tried debugging it, got into some ugly
cygwin-windows interactions, threw up, did the same you did and tried
to forget it all).

Alex Riesen wrote:

Alan Larkin, Sun, Apr 15, 2007 21:06:53 +0200:

quoted

quoted

quoted

Its not a huge push Im trying to do here (<about 150Mb) but
always malloc fails!

Any huge objects?

There were a couple of big files. I removed a 72Mb one (making 47Mb
the biggest one left in the project) and made the push and it
worked. I later pulled the project down to a different machine,
added the 72Mb file back in, and pushed to the server and it worked.
So apparently it's a platform specific problem. If anybody's
particularly interested I could replicate it under gdb and pass on
any info, but if not I wont ... job's done, Im happy.

72Mb is nothing. Is it Windows/cygwin, by any chance? (I see the
problem there sometimes. Tried debugging it, got into some ugly
cygwin-windows interactions, threw up, did the same you did and tried
to forget it all).

-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Thats what I thought, but malloc continued to fail until the 72Mb file was removed.

No, my server is running Debian. I had the push problem when logged in on that. It worked when I
pushed it from a Gentoo box.

On Mon, 16 Apr 2007, Alan Larkin wrote:

Alex Riesen wrote:

quoted

Alan Larkin, Sun, Apr 15, 2007 21:06:53 +0200:

quoted

quoted

quoted

Its not a huge push Im trying to do here (<about 150Mb) but
always malloc fails!

Any huge objects?

There were a couple of big files. I removed a 72Mb one (making 47Mb
the biggest one left in the project) and made the push and it
worked. I later pulled the project down to a different machine,
added the 72Mb file back in, and pushed to the server and it worked.
So apparently it's a platform specific problem. If anybody's
particularly interested I could replicate it under gdb and pass on
any info, but if not I wont ... job's done, Im happy.

72Mb is nothing. Is it Windows/cygwin, by any chance? (I see the
problem there sometimes. Tried debugging it, got into some ugly
cygwin-windows interactions, threw up, did the same you did and tried
to forget it all).

-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Thats what I thought, but malloc continued to fail until the 72Mb file was removed.

No, my server is running Debian. I had the push problem when logged in on that. It worked when I
pushed it from a Gentoo box.

a problem with ulimit -m on the server?

-- 
Julian

  ---
No skis take rocks like rental skis!

On 4/16/07, Alan Larkin [off-list ref] wrote:

quoted

72Mb is nothing. Is it Windows/cygwin, by any chance? (I see the
problem there sometimes. Tried debugging it, got into some ugly
cygwin-windows interactions, threw up, did the same you did and tried
to forget it all).

Thats what I thought, but malloc continued to fail until the 72Mb file was removed.

No, my server is running Debian. I had the push problem when logged in on that.
It worked when I pushed it from a Gentoo box.

Ok, now _that_ is interesting... Any limits on that server?
What git version?  There were some memory leaks fixed recently...

Julian Phillips wrote:

On Mon, 16 Apr 2007, Alan Larkin wrote:

quoted

Alex Riesen wrote:

quoted

Alan Larkin, Sun, Apr 15, 2007 21:06:53 +0200:

quoted

quoted

quoted

Its not a huge push Im trying to do here (<about 150Mb) but
always malloc fails!

Any huge objects?

There were a couple of big files. I removed a 72Mb one (making 47Mb
the biggest one left in the project) and made the push and it
worked. I later pulled the project down to a different machine,
added the 72Mb file back in, and pushed to the server and it worked.
So apparently it's a platform specific problem. If anybody's
particularly interested I could replicate it under gdb and pass on
any info, but if not I wont ... job's done, Im happy.

72Mb is nothing. Is it Windows/cygwin, by any chance? (I see the
problem there sometimes. Tried debugging it, got into some ugly
cygwin-windows interactions, threw up, did the same you did and tried
to forget it all).

-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Thats what I thought, but malloc continued to fail until the 72Mb file
was removed.

No, my server is running Debian. I had the push problem when logged in
on that. It worked when I
pushed it from a Gentoo box.

a problem with ulimit -m on the server?

Unlimited ...

On 4/16/07, Julian Phillips [off-list ref] wrote:

quoted

Thats what I thought, but malloc continued to fail until the 72Mb file was removed.

No, my server is running Debian. I had the push problem when logged in on that.
It worked when I pushed it from a Gentoo box.

a problem with ulimit -m on the server?

Nah, a process gets killed in case of a limit hit.
He gets malloc returning NULL.

Alex Riesen wrote:

On 4/16/07, Alan Larkin [off-list ref] wrote:

quoted

quoted

72Mb is nothing. Is it Windows/cygwin, by any chance? (I see the
problem there sometimes. Tried debugging it, got into some ugly
cygwin-windows interactions, threw up, did the same you did and tried
to forget it all).

Thats what I thought, but malloc continued to fail until the 72Mb file
was removed.

No, my server is running Debian. I had the push problem when logged in
on that.
It worked when I pushed it from a Gentoo box.

Ok, now _that_ is interesting... Any limits on that server?
What git version?  There were some memory leaks fixed recently...
-
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Ah, perhaps youre on to something there. The version Im running on the server is 1.4.4.4 (from apt
package manager), but on my own machine (from whence the push was successful) its 1.5.0.5. Still,
youd expect the problem to have effected more people if its a leak or something in the older version.

On 4/16/07, Alan Larkin [off-list ref] wrote:

quoted

quoted

quoted

72Mb is nothing. Is it Windows/cygwin, by any chance? (I see the
problem there sometimes. Tried debugging it, got into some ugly
cygwin-windows interactions, threw up, did the same you did and tried
to forget it all).

Thats what I thought, but malloc continued to fail until the 72Mb file
was removed.

No, my server is running Debian. I had the push problem when logged in
on that.
It worked when I pushed it from a Gentoo box.

Ok, now _that_ is interesting... Any limits on that server?
What git version?  There were some memory leaks fixed recently...

Ah, perhaps youre on to something there. The version Im running on the server is
1.4.4.4 (from apt package manager), but on my own machine (from whence the push
was successful) its 1.5.0.5. Still, youd expect the problem to have effected more
people if its a leak or something in the older version.

That's how the leaks were found: people were affected and complained :)
But it also a bit of unusual repo you have: not many have objects more
the 10Mb in their sources. Anyway, give the new version a try. Maybe it
was fixed, or maybe we have a chance to improve the current version.