Thread (224 messages) 224 messages, 7 authors, 2018-04-06

Re: Reduce pack-objects memory footprint?

From: Duy Nguyen <hidden>
Date: 2018-03-01 09:00:52

On Wed, Feb 28, 2018 at 06:22:33PM +0000, Eric Wong wrote:
Duy Nguyen [off-list ref] wrote:
quoted
which saves 12 bytes (or another 74 MB). 222 MB total is plenty of
space to keep some file cache from being evicted.
Nice!  I can definitely benefit from lower memory usage when
packing.  Fwiw, I use pahole with other projects to help find
packing opportunities:

	git://git.kernel.org/pub/scm/devel/pahole/pahole.git
Yes it's a wonderful tool.
quoted
@@ -14,11 +26,10 @@ struct object_entry {
 	void *delta_data;	/* cached delta (uncompressed) */
 	unsigned long delta_size;	/* delta data size (uncompressed) */
 	unsigned long z_delta_size;	/* delta data size (compressed) */
-	enum object_type type;
-	enum object_type in_pack_type;	/* could be delta */
 	uint32_t hash;			/* name hint hash */
-	unsigned int in_pack_pos;
 	unsigned char in_pack_header_size;
+	unsigned type:3;	 /* enum object_type */
+	unsigned in_pack_type:3; /* enum object_type - could be delta */
For C99 compilers, enums can be bitfields.  I introduced the
following macro into Ruby a few weeks ago to remain compatible
with non-C99 compilers:

/*
 * For declaring bitfields out of non-unsigned int types:
 *   struct date {
 *      BITFIELD(enum months) month:4;
 *      ...
 *   };
 */
#if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L)
# define BITFIELD(type) type
#else
# define BITFIELD(type) unsigned int
#endif
I tried this and got

In file included from builtin/pack-objects.c:20:0:
./pack-objects.h:49:19: l?i: ?type? is narrower than values of its type [-Werror]
  enum object_type type:TYPE_BITS;
                   ^~~~

The compiler is not wrong. What it does not realize is pack-objects
code never uses out-of-range values (OBJ_BAD and OBJ_ANY) but I don't
see how I could suppress this warning. So I went back to non-enum
bitfields.

--
Duy
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help