How do I stop git enumerating my working directory

Hi,

I'm a recent convert to git. I've been swapping over my projects
from cvs to git. The latest candidate for conversion is the
config files for my Linux server. I've been using cvs to record
all the config changes, and - mostly-automatically - to setup
new servers. Git will do a far better job, I think, but
I think I'm going to run into a problem: the root directory
of the server will be the working directory. Only relatively
few files will be under version control. How do I stop git
enumerating the whole drive whenever I do things like git-diff?

I don't think I can make .gitignore files do the job, because
it seems that you can set up to ignore a whole directory,
and then partially countermand that by placing a .gitignore
file (containing ! commands) inside the directory. That
makes me think that ignoring doesn't prevent the
enumeration.

Any help most appreciated. I really want to use git for
this.

Cheers,
	Paul.