Am 27.02.24 um 08:53 schrieb Jeff King:

On Mon, Feb 26, 2024 at 07:17:00PM +0100, René Scharfe wrote:

quoted

quoted

This is pulling heavily from strbuf_vaddf(). This might be a dumb idea,
but... would it be reasonable to instead push a global flag that causes
xmalloc() to use a memory pool instead of the regular heap?

Then you could do something like:

  push_mem_pool(pool);
  str = xstrfmt("%.*s~%d^%d", ...etc...);
  pop_mem_pool(pool);

It's a little more involved at the caller, but it means that it now
works for all allocations, not just this one string helper.

That would allow to keep track of allocations that would otherwise leak.
We can achieve that more easily by pushing the pointer to a global array
and never freeing it.  Hmm.

I see two uses for memory pools:

  1. You want to optimize allocations (either locality or per-allocation
     overhead).

  2. You want to make a bunch of allocations with the same lifetime
     without worrying about their lifetime otherwise. And then you can
     free them all in one swoop at the end.

And my impression is that you were interested in (2) here. If what
you're saying is that another way to do that is:

  str = xstrfmt(...whatever...);
  attach_to_pool(pool, str);

  ...much later...
  free_pool(pool);

then yeah, I'd agree. And that is a lot less tricky / invasive than what
I suggested.

quoted

It would not allow the shortcut of using the vast pool as a scratch
space to format the string with a single vsnprintf call in most cases.
Or am I missing something?

So here it sounds like you do care about some of the performance
aspects. So no, it would not allow that. You'd be using the vast pool of
heap memory provided by malloc(), and trusting that a call to malloc()
is not that expensive in practice. I don't know how true that is, or how
much it matters for this case.

In the specific use case we can look at three cases:

1. xstrfmt() [before 1c56fc2084]
2. size calculation + pre-size + strbuf_addf() [1c56fc2084]
3. mem_pool_strfmt() [this patch]

The performance of 2 and 3 is basically the same, 1 was worse.

xstrfmt() and strbuf_addf() both wrap strbuf_vaddf(), which uses
malloc() and vsnprintf().  My conclusion is that malloc() is fast
enough, but running vsnprintf() twice is slow (first time to determine
the allocation size, second time to actually build the string).  With a
memory pool we almost always only need to call it once per string, and
that's why I use it here.

The benefit of this patch (to me) is better code readability (no more
manual pre-sizing) without sacrificing performance.

The ability to clear (or UNLEAK) all these strings in one go is just a
bonus.

René