[RFC/RFH] Fun things with git-notes, or: patch tracking backwards

Hi everyone

I'll start with the fun first, try this in a git.git:

  git fetch git://repo.or.cz/git/trast.git mailnotes &&
  GIT_NOTES_REF=FETCH_HEAD git log origin/pu

I played around with some python code over the weekend that
automatically filters through git@vger.kernel.org history and scans
for patches, threads, and "What's cooking" messages.  So far it seems
to be working ok.

The net effect is that we get a backwards patch tracker: instead of
tracking the patches via some other means (say a web interface), the
automatic annotations can reconstruct more information about the
patches than what is eventually contained in the commits, and make it
available via git-notes.

Right now it only applies the patches that it finds, and associates
that with known commits to annotate them.  I eventually want to scan
for replies to patch text too, and insert them into the notes too.  I
also plan to publish topics for the patches somewhere (they're already
applied locally) and track at least a rudimentary status, perhaps one
of "unreplied", "replied" and "accepted".  (Distinguishing "rejected"
seems AI-complete.)

The whole process is a bunch of python scripts currently available at

  git://repo.or.cz/trackgit.git
  http://repo.or.cz/w/trackgit.git

It's all a bit after-the-fact right now, and can't cope with a few
things yet, for example patch series that aren't in git.git appearing
out of sequence to the mail reader.

Runtime is okay-ish so that I should be able to run it as a cronjob;
note regeneration is almost negligible (<1min), importing a month's
worth of mails takes on the order of 20min, and scanning history (to
know about the base blobs) since v1.6.0 is around 3min.

So the RFC bit is, is this useful to anyone?  What information would
you like to see in it, and what could be left out?

And the RFH: I don't have a full mail archive, not even since I joined
the list.  There also doesn't seem to be a convenient download button
on gmane.  Does anyone have (or know of) an archive going at least
back to v1.6.0 (not sure if any further back is interesting), which
was released in August last year, that you could send to me?

Thanks in advance :-)

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

Hi Thomas,

On Mon, Feb 09, 2009 at 03:08:08PM +0100, Thomas Rast wrote:

And the RFH: I don't have a full mail archive, not even since I joined
the list.  There also doesn't seem to be a convenient download button
on gmane.

you can download emails in mbox format from gmane by running

  wget http://download.gmane.org/gmane.comp.version-control.git/X/Y

which will download all emails starting at the email with "gmane id" X
and ending at the email with "gmane id" Y-1.

So, if you want to download the whole archive, you could run

  wget http://download.gmane.org/gmane.comp.version-control.git/1/109090

(but of course that 109090 will be larger by the time you will reading
this).  However, there's a catch, as gmane's script execution time is
limited to 30 sec, so you will not get the whole archive, but only the
first couple of thousands emails (in my case and network I got around
6k emails in one run).  Therefore you'll need some loop to download
only a few thousand emails in each iteration.


Best,
Gábor

SZEDER Gábor wrote:

you can download emails in mbox format from gmane by running

  wget http://download.gmane.org/gmane.comp.version-control.git/X/Y

which will download all emails starting at the email with "gmane id" X
and ending at the email with "gmane id" Y-1.

Thanks, that will do nicely!

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

Thomas Rast wrote:

  git fetch git://repo.or.cz/git/trast.git mailnotes &&
  GIT_NOTES_REF=FETCH_HEAD git log origin/pu

An update: I have fully automated the process, it now fetches mails
from Gmane over HTTP which gives it the Gmane URLs for free.  I'm
rather happy with the latter feature, especially since Konsole has a
feature to recognize and open links directly.

I have imported all commits, and mails since roughly July 2008
(starting with Gmane 89000).  In this timeframe there were 1802
non-merge commits, and the mailnotes tree now holds 1122 annotations.
I won't import mails any further back until the parsing-related code
has become reasonably stable, but this at least covers the post-v1.6.0
commits.

-- 
Thomas Rast
trast@{inf,student}.ethz.ch

On Sunday 08 February 2009 20:25, Shawn O. Pearce wrote:

bill lam [off-list ref] wrote:

quoted

I track /etc using a config

[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	worktree = /etc
	logAllRefUpdates = true
	excludesfile =

But that can not be handled by tools,

git gui : cannot use funny .git directory .

If someone sends patches for git-gui, maybe.  This use case of
different repository and worktree isn't very common for git-gui
so it doesn't support it.

I came across the same problem. I also sent a tentative patchset to
solve the problem about 12 hours ago:

http://thread.gmane.org/gmane.comp.version-control.git/109035

-- 
Giuseppe "Oblomov" Bilotta