Thread (37 messages) 37 messages, 3 authors, 2025-07-15
STALE368d

[PATCH 0/8] odb: track multi-pack-indices via their object sources

From: Patrick Steinhardt <hidden>
Date: 2025-07-09 07:55:19

Hi,

multi-pack-indices are tracked via `struct multi_pack_index`. This data
structure is stored inside `struct object_database`, which is the global
database that spans across all of the object sources

This layout causes two problems:

  - Multi-pack indices aren't global to an object database, but instead
    there can be one multi-pack index per object source. This creates a
    mismatch between the on-disk layout and how things are organized in
    the object database subsystems and makes some parts, like figuring
    out whether an object source has an MIDX, quite awkward.

  - Multi-pack indices are an implementation detail of how efficient
    access for packfiles work. As such, they are neither relevant in the
    context of loose objects, nor in a potential future where we have
    pluggable backends.

This patch series thus refactors the codebase to stop tracking MIDX's
globally. Instead, they are being pushed down one level so that every
`struct odb_source` has an optional MIDX itself. This simplifies some of
our code and will make it easier in a future iteration to move the data
into a packfile-specific object source backend.

This series is built on top of a30f80fde92 (The eighth batch,
2025-07-08) with "ps/object-store" at 841a03b4046 (odb: rename
`read_object_with_reference()`, 2025-07-01) merged into it.

Thanks!

Patrick

---
Patrick Steinhardt (8):
      midx: start tracking per object database source
      packfile: refactor `prepare_packed_git_one()` to work on sources
      midx: stop using linked list when closing MIDX
      midx: track whether we have loaded the MIDX
      packfile: refactor `get_multi_pack_index()` to work on sources
      packfile: stop using linked MIDX list in `find_pack_entry()`
      packfile: stop using linked MIDX list in `get_all_packs()`
      midx: remove now-unused linked list of multi-pack indices

 builtin/pack-objects.c |  9 ++++--
 builtin/repack.c       |  4 +--
 midx-write.c           | 22 ++-----------
 midx.c                 | 38 +++++++++-------------
 midx.h                 |  9 +++---
 object-name.c          | 21 ++++++++-----
 odb.h                  | 19 +++++------
 pack-bitmap.c          | 20 ++++++++----
 packfile.c             | 85 ++++++++++++++++++++++----------------------------
 packfile.h             |  3 +-
 10 files changed, 105 insertions(+), 125 deletions(-)


---
base-commit: 1ad11c8963b6902bb6a98f5071591ad358329fe4
change-id: 20250513-b4-pks-midx-via-odb-alternate-d4b5940a28cd
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help