how about this as a basis for git-annotate?

#!/bin/bash

file="$1"
anno="$file.anno"
prev=""
export GIT_DIFF_OPTS=-u0

rm $anno
touch $anno

revs=`git-rev-list HEAD | tac`

for rev in $revs; do
	if [ ! -z "$prev" ]; then
		git-diff-tree -p $prev $rev $file \
		| sed 	-e "/^--- /p" \
			-e "/^+++ /p" \
			-e "/^--- /d" \
			-e "/^+++ /d" \
			-e "s/^-/-$prev /g" \
			-e "s/^+/+$rev /g" | patch $anno
	fi
	prev=$rev
done



* gives a warning when the diff is empty
* has a problem with "\ No newline at end of file"

What I tried to do is to reconstruct the file from the diffs,
and prefixing each line with the sha1 of the revision.

After further testing, I see that the implementation I provided does not 
work,
because the context is not necessarily from the previous version, but from 
/a/ previous version.
Still the idea should work.
So patch needs to ignore the sha1 of the revision.
Maybe git-apply can be modified to do this...?