On Thu, May 08, 2025 at 10:24:55AM +0000, Abhishek Dalmia wrote:

I ran into an edge case while testing incremental backups with git
bundle. If a commit is created with a timestamp earlier than the
latest full or incremental backup, it can be excluded from the next
bundle due to the --since parameter even if there is a buffer.

Yeah, I don't think you want to use "--since" here, since it is about
commit timestamps. You care about the state of the refs at a particular
time. Or more accurately, you care that you have captured a particular
ref state previously.

So ideally you'd snapshot that state in an atomic way, feed it as the
"current" state when doing a bundle, and then save it for later. You can
easily create such a snapshot with for-each-ref, but I don't think
git-bundle has a way to provide the exact set of ref tips and their
values (it just takes rev-list arguments, and wants to resolve the refs
themselves).

You could probably get away with just creating a bundle with the current
state, and then pulling the snapshot values from the created bundle.
Something like this:

  # for initial backup
  if ! test -e last-bundle-snapshot; then
    >last-bundle-snapshot
  fi

  # mark everything from last as seen, so we do not include it,
  # along with --all (or your choice of refs) to pick up everything
  # we have currently
  sed -e 's/^/^/' <last-bundle-snapshot |
  git bundle create out.bundle --all --stdin

  # and now save that ref state for next time; this is inherently
  # peeking at the bundle format.
  sed -ne '
	# quit when we see end of header
	/^$/q;
	# drop comments and old negatives; copy only first word (the oid)
	s/^\([^-#][^ ]*\).*/\1/p;
  ' <out.bundle >last-bundle-snapshot

Or alternatively, instead of using git-bundle at all, you could just
store a collection of ref snapshots (from "for-each-ref") and thin packs
(from "pack-objects --thin --stdout", fed from the old snapshot and the
new). Which is really all that bundles are anyway.

-Peff