Re: [PATCH v4 08/23] ewah: compressed bitmap implementation

On Wed, Jan 22, 2014 at 06:05:36PM -0800, Jonathan Nieder wrote:

Jeff King wrote:

quoted

EWAH is a word-aligned compressed variant of a bitset (i.e. a data
structure that acts as a 0-indexed boolean array for many entries).

I suspect that for some callers it's not word-aligned.

Yes, the mmap'd buffers aren't necessarily word-aligned. I don't think
we can fix that easily without changing the on-disk format (which comes
from JGit anyway). However, since we are memcpying the bulk of the data
into a newly allocated buffer (which must be aligned), we can do that
first, and then fix the endian-ness in place.

The only SPARC machine I have access to is running Solaris, but after
some slight wrestling with the BYTE_ORDER macros, I managed to get it to
compile and reproduced the bus error.

Here's a patch series (on top of jk/pack-bitmap, naturally) that lets
t5310 pass there. I assume the ARM problem is the same, though seeing
the failure in realloc() is unexpected. Can you try it on both your
platforms with these patches?

  [1/2]: compat: move unaligned helpers to bswap.h
  [2/2]: ewah: support platforms that require aligned reads

Hopefully it's possible to get the alignment right in the caller
and tweak the signature to require that instead of using unaligned
reads like this.  There's still something wrong after this patch ---
the new result is a NULL pointer dereference in t5310.7 "enumerate
--objects (full bitmap)".

After my patches, t5310 runs fine for me. I didn't try your patch, but
mine are similar. Let me know if you still see the problem (there may
simply be a bug in yours, but I didn't see it).

-Peff