Re: Cygwin can't handle huge packfiles?

Nicolas Pitre [off-list ref] writes:

On Fri, 7 Apr 2006, Junio C Hamano wrote:

quoted

Linus Torvalds [off-list ref] writes:

quoted

On Mon, 3 Apr 2006, Linus Torvalds wrote:

quoted

That said, I think git _does_ have problems with large pack-files. We have 
some 32-bit issues etc

I should clarify that. git _itself_ shouldn't have any 32-bit issues, but 
the packfile data structure does. The index has 32-bit offsets into 
individual pack-files. 

That's not hugely fundamental,...

Linus _does_ understand what he means, but let me clarify and
outline a possible future direction.

For the record, the delta code also has 32-bit limitations of its own 
presently.  It cannot encode a delta against a buffer which is larger 
than 4GB.

I however made sure the byte 0 could be used as a prefix for future 
encoding extensions, like 64-bit file offsets for example.

True the delta data representation, not just the "delta code",
has that limitation, but I do not think you issue "insert 0-byte
literal data" command from the deltifier side right now, so we
should be OK.

Maybe we would want to check (cmd == 0) case to detect delta
extension that we do not handle right now?

On Fri, 7 Apr 2006, Junio C Hamano wrote:

Nicolas Pitre [off-list ref] writes:

quoted

On Fri, 7 Apr 2006, Junio C Hamano wrote:

quoted

Linus Torvalds [off-list ref] writes:

quoted

On Mon, 3 Apr 2006, Linus Torvalds wrote:

quoted

That said, I think git _does_ have problems with large pack-files. We have 
some 32-bit issues etc

I should clarify that. git _itself_ shouldn't have any 32-bit issues, but 
the packfile data structure does. The index has 32-bit offsets into 
individual pack-files. 

That's not hugely fundamental,...

Linus _does_ understand what he means, but let me clarify and
outline a possible future direction.

For the record, the delta code also has 32-bit limitations of its own 
presently.  It cannot encode a delta against a buffer which is larger 
than 4GB.

I however made sure the byte 0 could be used as a prefix for future 
encoding extensions, like 64-bit file offsets for example.

True the delta data representation, not just the "delta code",
has that limitation, but I do not think you issue "insert 0-byte
literal data" command from the deltifier side right now, so we
should be OK.

Maybe we would want to check (cmd == 0) case to detect delta
extension that we do not handle right now?

Good idea.  Will send you a patch.


Nicolas