Re: space compression (again)

On Fri, Apr 15, 2005 at 02:45:55PM -0400, C. Scott Ananian wrote:

quoted

- we already have wasted space due to the low-level filesystem (as
  opposed to "git") usually being block-based, which means that space
  utilization for small objects tends to suck. So you really want to
  prefer objects that are several kB (compressed), and a small block just
  wastes tons of space.

Not on (say) reiserfs, and not over the network.  I'm proposing (at the 
moment) easy conversion from chunked to unchunked disk representation,
so that you can leave things unchunked if (for example) you know you're 
running ext2 with a large block size.

Or if one does not care about space,  and simply want's speed,  add another
layer of indirection - a flattened container object which has hashses as
normal,  then as it's content simply has the 'chunk list object' and the
'chunk objects' concatenated.

It's then a per user / database as to if the flattened objects,  or the
heirarcal objects are storred locally.

DF