OK, I would say that this failing is semi-expected. :) The caching
system works using "git notes", which are stored in refs in the
repository. And since you are running "diff --no-index" outside of a
repository, there is nowhere to put them.

I have not mentioned this specifically, but my goal is a general diff
command, which internally uses text conversions, pager etc. as
configured for git.
It makes sense to cache the textconv results when used in a
repository, but I don't think it should fail when not in one.
I think the default behavior should be to silently skip caching in
such situations but produce a diff otherwise.

Of course any time we see a BUG something has gone wrong. What I think
_should_ happen is that we should quietly disable the caching (which,
after all, is just an optimization) and otherwise complete the command.

In my script I currently disable caching explicitly for all drivers:

keys=$(git config --name-only --get-regexp '^diff\.\w+\.cachetextconv$')
config=(); for key in $keys; do config+=(-c "$key=false"); done

git "${config[@]}" diff --no-index --no-prefix "$@"

But it seems like something git should handle on its own, so that diff
would accommodate use in different circumstances with the same config.

-Pawel