Johannes Schindelin [off-list ref] wrote:

Hi Eric,

On Tue, 26 Nov 2019, Eric Wong wrote:

quoted

Johannes Schindelin [off-list ref] wrote:

quoted

On Tue, 26 Nov 2019, Eric Wong wrote:

quoted

Johannes Schindelin [off-list ref] wrote:

quoted

The biggest obstacle is that at least one of those Pipelines requires
access to a clone of public-inbox.org/git, and cloning that is rather
expensive. Even a shallow fetch would be super expensive, by virtue of
_all_ the mails being blobs reachable from the tip commit's tree.

Fwiw, lore.kernel.org/git/$EPOCH.git ought to be somewhat cheaper,
but it's a different (more scalable) format which requires SQLite:

	https://public-inbox.org/public-inbox-v2-format.html

Is this incremental? GitGitGadget needs this to be incremental ;-)

Incremental as far as "git fetch" goes?  Of course :>
The "m" file is overwritten with every commit, so the tree size
stays at 1 (tree growth was a major scalability problem in v1).

Let me try again:

GitGitGadget "reads" the mail via the incremental clone, remembering the
hash of the latest processed commit. When the Azure Pipeline runs, it
first fetches, and if the commit is still the same, does nothing but exit
with success. If the commit is different, it looks at the mails that were
added, via `git log -p <previous-tip-commit>..<tip-commit>`.

Is that possible with the v2 format?

Of course, yes.  The Xapian and SQLite indexing also works the
same way "git log prev..tip" and storing the latest commit hash.