Re: [FAQ?] Rationale for git's way to manage the index

Linus Torvalds [off-list ref] writes:

On Sun, 6 May 2007, Matthieu Moy wrote:

quoted

But the fact that git actually remembers the _content_ of files in the
index, and that the default behavior for "commit" is to commit only
the content that is explicitely "git add"ed is something I've never
seen outside git.

Yeah. You'd better get used to it, because it's fundamental.

Thanks a lot for the detailed explanations.

Note that I'm not "complaining", but just not understanding something.

(I would actually complain about the documentation not being clear
enough, but I'll try to complain with a contribution instead ;-) I'll
add something to the FAQ on the wiki, but it's down right now).

 - You fundamentally cannot do it any other way.

   Not doing it the way git does it (point to the content) means that the 
   index-replacement has to point to something else, namely a "file ID". 

Well, git's index still tells more than "the content FOOBAR exists,
somewhere". It also "contains", if not "points to", the file name.

What's so hard with adding that "-a" to "git commit"? You don't even need 
it on the status line, the status is relevant and understandable (and 
actually tells you more) even without it.

Off course, I don't have strong argument against it. The biggest
annoyance is that my fingers are used to "commit -m message", and now
type "commit -a message", but ...

The reason why I'm posting this is that I was wondering whether
"commit -a" not being the default was supposed to be a message like
"you shouln't use it too often".

It seems it isn't. I'll just get used to "commit -a" (and probably
alias it), and discover the actual benefits of the index little by
little.

[...] it basically could be used ass a definition of CVS: [...]

                                   ^^^
Not sure this was intentional, but your spelling of "as" when used to
talk about CVS seems to reveal something about your state of mind ;-).

Thanks,

-- 
Matthieu

On Sun, 6 May 2007, Matthieu Moy wrote:

Well, git's index still tells more than "the content FOOBAR exists,
somewhere". It also "contains", if not "points to", the file name.

Indeed. 

Git's index is basically very much defined as

 - sufficient to contain the total "content" of the tree (and this 
   includes all metadata: the filename, the mode, and the file contents 
   are all *parts* of the "content", and they are all meaningless on their 
   own!)

 - additional "stat" information to allow the obvious and trivial (but 
   hugely important!) filesystem comparison optimizations.

So you really should see it as *being* the content. The content is not the 
"file name" or the "file content" as separate parts. You really cannot 
separate the two. Filenames on their own make no sense (they have to have 
file content too), and file content on its own is similarly senseless (you 
have to know how to reach it).

What I'm trying to say is that git fundmaentally doesn't _allow_ you to 
see a filename without its content. The whole notion is insane and not 
valid. It has no relevance for "reality".

Also, you should realize that when you do

	git add X

you are *not* adding the filename X. No, "X" is literally a "content path 
pattern", the same way it is when you do something like

	gitk X

and it's worth always keeping in mind that in neither case is "X" 
necessarily a single file, but literally a pathname pattern that is used 
as a "filter" on all the possible patterns.

(Of course, the filtering rules are different for "git add" and "gitk": in 
the "git add" example, you filter the working tree files, while in "gitk" 
you filter the files that git already knows about, so they are different, 
but in both cases you really should think of them as filters, not as 
"filenames", even though one _trivial_ filter is to give a filter that 
matches exactly one pathname).

The reason why I'm posting this is that I was wondering whether
"commit -a" not being the default was supposed to be a message like
"you shouln't use it too often".

No, "git commit -a" is undoubtedly _convenient_. You can use it as often 
as you like.

So as long as you see it as a convenience feature, and realize that "git 
commit" is actually a lot more powerful than just being able to always do 
the convenient, go on and use "git commit -a" all the time.

When you hit a situation where you want to do something slightly subtler, 
you'll suddenly be really happy that you always had the convenience 
feature, but that git didn't make you think that it was how you _had_ to 
work.

quoted

[...] it basically could be used ass a definition of CVS: [...]

                                   ^^^
Not sure this was intentional, but your spelling of "as" when used to
talk about CVS seems to reveal something about your state of mind ;-).

Indeed ;)

Freudian slip. But yes, I'm really down on CVS. The only thing I like less 
than CVS is SVN, and that's just because I think it's such a sad waste, 
not because it's actually _worse_ than CVS. (Ie I dislike SVN from a "it 
could have been so much better" perspective).

			Linus

On Sun, 6 May 2007, Matthieu Moy wrote:

The reason why I'm posting this is that I was wondering whether
"commit -a" not being the default was supposed to be a message like
"you shouln't use it too often".

Well, personally I practically never use it, I find that having a 
separation between what the current state of my tree is and what will be 
comitted to be one of the really "oh wow, why doens't everything else do 
this?" features.  However, i tend to be working on more than one thing at 
once, and switch between them - so I commit work on A while work on B is 
still unfinished, then start C, finish B some point later and commit it, 
and then I can finish C.  Git is the first VCS that supports a butterfly 
mind :P.

It seems it isn't. I'll just get used to "commit -a" (and probably
alias it), and discover the actual benefits of the index little by
little.

"git add -i" - this is a feature I have wanted since I started using 
version control ...

-- 
Julian

  ---
Your good nature will bring you unbounded happiness.

Hi,

On Sun, 6 May 2007, Matthieu Moy wrote:

Linus Torvalds [off-list ref] writes:

quoted

 - You fundamentally cannot do it any other way.

   Not doing it the way git does it (point to the content) means that the 
   index-replacement has to point to something else, namely a "file ID". 

Well, git's index still tells more than "the content FOOBAR exists,
somewhere". It also "contains", if not "points to", the file name.

As you pointed out yourself, the index _has_ an idea of the content of 
that file. So, arguably, it does not point to _that_ file, but rather to 
that file _with a certain content_.

quoted

What's so hard with adding that "-a" to "git commit"? You don't even need 
it on the status line, the status is relevant and understandable (and 
actually tells you more) even without it.

Off course, I don't have strong argument against it. The biggest
annoyance is that my fingers are used to "commit -m message", and now
type "commit -a message", but ...

Just another reason to hate CVS. Because it trained people to do that. If 
it was not for the training by CVS, I would have strongly opposed to the 
introduction of the "-m" switch to commit. It _encourages_ bad commit 
messages.

Now, with Git I usually let git-commit start up the editor. Because then I 
am actually encouraged to make up my mind, and put down a meaningful 
message, which might not only help _others_ to understand why I did it, 
and how, but also _myself_ (after a few months).

The reason why I'm posting this is that I was wondering whether "commit 
-a" not being the default was supposed to be a message like "you 
shouln't use it too often".

IMHO yes, that is the message.

In addition to being nice to people used to the behaviour of "git commit" 
_without_ other arguments.

Ciao,
Dscho

On 2007-05-06 23:53:13 +0100, Julian Phillips wrote:

On Sun, 6 May 2007, Matthieu Moy wrote:

quoted

The reason why I'm posting this is that I was wondering whether
"commit -a" not being the default was supposed to be a message
like "you shouln't use it too often".

Well, personally I practically never use it, I find that having a
separation between what the current state of my tree is and what
will be comitted to be one of the really "oh wow, why doens't
everything else do this?" features. However, i tend to be working on
more than one thing at once, and switch between them - so I commit
work on A while work on B is still unfinished, then start C, finish
B some point later and commit it, and then I can finish C. Git is
the first VCS that supports a butterfly mind :P.

git-gui is really handy for adding/committing a subset of the changes
in your working tree. Especially for those of us with goldfish memory,
since it's so easy to see exactly what's happening: what's going to be
committed and what not.

"git add -i" - this is a feature I have wanted since I started using
version control ...

I thought "git add -i" was the best thing since sliced bread -- until
I found the same feature in git-gui, but with a _much_ better
interface. Just right-click on a hunk in a diff, and you have the
option of staging/unstaging that hunk. Pure magic.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Karl Hasselstr??m [off-list ref] wrote:

I thought "git add -i" was the best thing since sliced bread -- until
I found the same feature in git-gui, but with a _much_ better
interface. Just right-click on a hunk in a diff, and you have the
option of staging/unstaging that hunk. Pure magic.

"git add -i" has a hunk splitting feature that git-gui lacks.
I'm thinking of adding features to git-gui to let you select a
region of a hunk using the text selection, and then stage only
that selection.  I also want to let you revert hunks from the
working directory copy.

But after reading Junio's comments about "git add -i" being a
possibly bad idea and instead letting you park everything into
a shelf, reset --hard your working directory to HEAD and then
pull things back off the shelf to be staged, I might want to
do that differently in git-gui...  like use a shelf.  ;-)


But I'm glad someone else finds the hunk feature useful in
git-gui.  I use it far too often myself.

-- 
Shawn.

On 5/7/07, Matthieu Moy [off-list ref] wrote:

Linus Torvalds [off-list ref] writes:

quoted

On Sun, 6 May 2007, Matthieu Moy wrote:

quoted

But the fact that git actually remembers the _content_ of files in the
index, and that the default behavior for "commit" is to commit only
the content that is explicitely "git add"ed is something I've never
seen outside git.

Yeah. You'd better get used to it, because it's fundamental.

Thanks a lot for the detailed explanations.

Heh. Making the index very visible makes sense when you are merging,
Linus and Junio are both integrators and spend a lot of time merging.
Hence the default is for git-commit to observe the index.

I agree with Linus' other points too, but at the end of the day, it
makes life easier and saner mainly when merging, at the expense of
having to pay a bit more attention in common commits. The tradeoff
makes sense _specially_ if you are the integrator.

So I do git-commit -a, and typing that '-a' is small price to pay for
the best SCM I've ever used ;-)

cheers,


martin

On Tue, 8 May 2007, Martin Langhoff wrote:

Heh. Making the index very visible makes sense when you are merging,
Linus and Junio are both integrators and spend a lot of time merging.
Hence the default is for git-commit to observe the index.

It is definitely true that some of the advantages of the way git does the 
index really start shinign when merging and you have content conflicts. 
What we've done to "git diff" really makes things a lot easier (and 
anybody who hasn't used "gitk --merge" after a content conflict really 
hasn't realized how *helpful* git is when merging content conflicts).

However, in all honesty, while the whole "index for merges" comes from 
pretty damn early in git history (the whole "stage number" thing appeared 
on April 15th 2005 - so it was about a week after the first release), it 
wasn't the original impetus of the way git works.

Git used explicit index updates from day 1, even before it did the first 
merge. It's simply how I've always worked. I tend to have dirty trees, 
with some random patch in my tree that I do *not* want to commit, because 
it's just a Makefile update for the next version (to remind me - I've 
released kernel versions too many times with an old version number, just 
because I forgot to update the Makefile).

Or other things like that - I have small test-patches in my tree that I 
want to build, but that I don't want to commit, and I end up doing big 
merges and whole patch-application sequences with such a dirty tree 
(obviously if the patch or merge wants to change that file, I then need to 
do something about that dirty state, but it happens surprisingly seldom).

So the whole "update stuff to be committed explicitly" ends up _really_ 
shining during a merge, but it actually is how I do non-merge development 
too.

			Linus

On 5/8/07, Linus Torvalds [off-list ref] wrote:

On Tue, 8 May 2007, Martin Langhoff wrote:

quoted

Heh. Making the index very visible makes sense when you are merging,
Linus and Junio are both integrators and spend a lot of time merging.
Hence the default is for git-commit to observe the index.

It is definitely true that some of the advantages of the way git does the
index really start shinign when merging and you have content conflicts.
What we've done to "git diff" really makes things a lot easier (and
anybody who hasn't used "gitk --merge" after a content conflict really
hasn't realized how *helpful* git is when merging content conflicts).

Totally, when merging git's approach is incredibly useful. gitk
--merge and the resolved conflicts not appearing in the default git
diff is great stuff.

For for small, simpleminded and mostly-linear development it's not
that important. Of course, I use git on projects large and small, so I
can understand it. For someone using it with a small mostly-linear
project, the whole index thing is overkill, and the explanations
pointless. I can understand people wondering WTF.

So the whole "update stuff to be committed explicitly" ends up _really_
shining during a merge, but it actually is how I do non-merge development
too.

On a large project it's always a good idea to commit with explicit
paths -- regardless of your SCM. As it happens, I have to use explicit
paths with CVS, or it'll punish me by taking solid minutes to do a 2
file commit. I am sure that the mozilla and OpenOffice developers
using CVS also commit with explicit paths. Life's too short to waste
an hour.

(The times are from working on Moodle, hosted on SF.net with ~4K
files, 700 directories.).

cheers,


m

"Shawn O. Pearce" wrote:

But I'm glad someone else finds the hunk feature useful in
git-gui.  I use it far too often myself.

It it among the most-wanted features here. We discovered it only because
Karl mentioned it yesterday. ;)

-- Hannes

On 2007-05-07 21:41:14 -0400, Shawn O. Pearce wrote:

Karl Hasselström [off-list ref] wrote:

quoted

I thought "git add -i" was the best thing since sliced bread --
until I found the same feature in git-gui, but with a _much_
better interface. Just right-click on a hunk in a diff, and you
have the option of staging/unstaging that hunk. Pure magic.

"git add -i" has a hunk splitting feature that git-gui lacks. I'm
thinking of adding features to git-gui to let you select a region of
a hunk using the text selection, and then stage only that selection.

That would be useful. It's currently possible to split some hunks by
reducing the number of content lines, but if the changes aren't
separated by any unchanged lines at all, that doesn't work.

I also want to let you revert hunks from the working directory copy.

That would be handy. But unlike stage/unstage, this can lose
information, so there'd need to be some kind of "are you _really_
sure? [Yes] [No]" safety hatch, which would make it less convenient.

But after reading Junio's comments about "git add -i" being a
possibly bad idea and instead letting you park everything into a
shelf, reset --hard your working directory to HEAD and then pull
things back off the shelf to be staged, I might want to do that
differently in git-gui... like use a shelf. ;-)

A shelf could be handy. Actually, it could be handy to have more than
one. Then one could go through the mess in one's working directory and
toss changes into one bin for each commit one plans to create --
including one "trash" bin for hunks one would like to revert.

I assume that shelves would be implemented as branches that are
precisely one commit on top of HEAD? If so, I'd just like to point out
that they're exactly like unapplied patches in StGIT.

Hmm. I find it inconsistent to force or strongly encourage the user to
commit precisely the working directory changes and not a subset
thereof, which the shelf idea seems to encourage, while at the same
time not committing straight from the working directory but from a
specific staging area (the index).

But I'm glad someone else finds the hunk feature useful in git-gui.
I use it far too often myself.

I don't think it's a bad thing. If I've made several unrelated changes
and want to commit them separately for the sake of readable history,
how exactly is that a bad thing when compared to committing it all at
once? If I care about clean history in the first place, then
presumably I'll test the commits in isolation if I deem it necessary
-- and if I don't, then I probably won't test anyway even if the tool
makes it easy.

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

On 2007-05-08 09:15:45 +0200, Johannes Sixt wrote:

"Shawn O. Pearce" wrote:

quoted

But I'm glad someone else finds the hunk feature useful in
git-gui. I use it far too often myself.

It it among the most-wanted features here. We discovered it only
because Karl mentioned it yesterday. ;)

See? Who said spamming doesn't work? :-)

I think it would be worth introducing git-gui as a commit tool in the
tutorial(s) and the manual. It gives a very nice graphical
representation of the dirty state you're going to commit, and the
dirty state you aren't going to commit because you haven't staged it
yet. The only drawback is that it's a lot of work to make
documentation with screenshots ...

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

Hi,

On Tue, 8 May 2007, Martin Langhoff wrote:

Heh. Making the index very visible makes sense when you are merging,

You're saying that the main use of the index is to help merging. I have to 
disagree strongly.

When I have been chasing a bug all over the place, and finally found it, 
my working tree is a mess. Lots of assertions, lots of debugging 
statements, some of them commented out. So, now it is cleanup time, right?

The problem is that more often than not, I broke my fix while cleaning up.

Therefore, I now put all changed files into the index (git add -u), and 
clean up the files one by one, always checking with "git diff" and "git 
diff HEAD" what I still have to do.

Yes, very often I can just take the original version of a file (git reset 
--soft <file...> would be handy here), but it helped me quite a number of 
times to have my messed-up-but-working state in the index.

In a sense, I am using the index as the stash commit we talked about every 
once in a while.

Ciao,
Dscho

On Tue, May 08, 2007 at 12:28:36PM +0200, Karl Hasselström wrote:

I think it would be worth introducing git-gui as a commit tool in the
tutorial(s) and the manual. It gives a very nice graphical
representation of the dirty state you're going to commit, and the
dirty state you aren't going to commit because you haven't staged it
yet. The only drawback is that it's a lot of work to make
documentation with screenshots ...

You could put that on a web page someplace.

For the tutorial and user manual, could git-gui be treated similar gitk,
with just a one- or two- line mention here and there?  I haven't used
it, so don't know where it would most logically fit in....

--b.

Karl Hasselstr??m [off-list ref] wrote:

It's currently possible to split some hunks by
reducing the number of content lines, but if the changes aren't
separated by any unchanged lines at all, that doesn't work.

Yea, I've played that game before too (reduce content lines) to
try and simulate a hunk splitter.  ;-)  Doesn't always work.

Right now I feel like a huge chunk of the git-gui code is simply not
maintainable.  The 0.7.0 release is really more about refactoring the
code to make it more maintainable, than it is about actual features
(though there are some new things, like vi-keys).

The hunk selection stuff is just one part of the 2,000 lines
still left in git-gui.sh itself, and that still uses a lot of
messy globals.  I want to get the code better organized before
I take on major new additions to it.

quoted

I also want to let you revert hunks from the working directory copy.

That would be handy. But unlike stage/unstage, this can lose
information, so there'd need to be some kind of "are you _really_
sure? [Yes] [No]" safety hatch, which would make it less convenient.

True, but that beats the tar out of copying the - lines to your
clipboard and pasting them into your text editor, then deleting
the - prefix.  Especially if its a couple of hunks that you want
to revert.  Which I find myself doing all to often.

Actually I work around it today by staging what I care about,
then reverting the file.  Since the revert comes out of the index,
I get (mostly) the same action as reverting a particular hunk.
But it does mean that I lose my index state, if that happened to
be of any particular interest.

I assume that shelves would be implemented as branches that are
precisely one commit on top of HEAD? If so, I'd just like to point out
that they're exactly like unapplied patches in StGIT.

I haven't looked at StGIT in a while.  I've seen noise on the list
about nifty features being added, but I haven't kept up with what
those features actually are.  I think you are right about this and
maybe git-gui should try to be compatible with StGIT's unapplied
patches, should I get into actually implementing a shelving system.
 

Hmm. I find it inconsistent to force or strongly encourage the user to
commit precisely the working directory changes and not a subset
thereof, which the shelf idea seems to encourage, while at the same
time not committing straight from the working directory but from a
specific staging area (the index).

Indeed; I was thinking that this very morning.  Making an index that
you stage things into, but then also saying you cannot really do that
and instead have to shelve what you don't want - that's just evil.
I'll have to think about it more.

The blame interface in git-gui needs help more than the index
staging features.  The colors suck.  ;-)
 
-- 
Shawn.

On 2007-05-08 08:40:27 -0400, J. Bruce Fields wrote:

On Tue, May 08, 2007 at 12:28:36PM +0200, Karl Hasselström wrote:

quoted

I think it would be worth introducing git-gui as a commit tool in
the tutorial(s) and the manual. It gives a very nice graphical
representation of the dirty state you're going to commit, and the
dirty state you aren't going to commit because you haven't staged
it yet. The only drawback is that it's a lot of work to make
documentation with screenshots ...

For the tutorial and user manual, could git-gui be treated similar
gitk, with just a one- or two- line mention here and there? I
haven't used it, so don't know where it would most logically fit
in....

I would introduce it with a paragraph or two right where committing is
covered the first time. Explain that the empty file list box to the
left contains the changes that will be committed when you press the
commit button, and that the file list box on the right contains the
changes that won't be committed. By clicking on a file name you get to
see the diff to the file, and by clicking on the icon you move it to
the other file list box -- that is, you stage/unstage it.

And now comes the clever part: Introduce the index, by explaining that
it essentially _is_ the left file list box. Explain that git-add is
the command-line equivalent of moving changes to the left box, and
that git-commit without arguments simply commits what's in the index
-- exactly like git-gui's Commit button.

I think it could work. :-)

-- 
Karl Hasselström, kha@treskal.com
      www.treskal.com/kalle

On Tue, May 08, 2007 at 04:53:11PM +0200, Karl Hasselström wrote:

I would introduce it with a paragraph or two right where committing is
covered the first time. Explain that the empty file list box to the
left contains the changes that will be committed when you press the
commit button, and that the file list box on the right contains the
changes that won't be committed. By clicking on a file name you get to
see the diff to the file, and by clicking on the icon you move it to
the other file list box -- that is, you stage/unstage it.

And now comes the clever part: Introduce the index, by explaining that
it essentially _is_ the left file list box. Explain that git-add is
the command-line equivalent of moving changes to the left box, and
that git-commit without arguments simply commits what's in the index
-- exactly like git-gui's Commit button.

I think it could work. :-)

Definitely, sounds fun.

For the in-tree documentation, maybe I'm just my crusty text-centric
commandline point of view, but I'd rather have the primary explanation
continue to depend only on text and commandline examples, and then add a
note telling people that playing with git-gui may help develop their
intuition for the way the index works.

But I think it'd be interesting to try out the above approach with
screenshots, etc., on a web page someplace.  It might also make a good
visual aid for a talk.

--b.

Hi,

On Tue, 8 May 2007, J. Bruce Fields wrote:

On Tue, May 08, 2007 at 04:53:11PM +0200, Karl Hasselström wrote:

quoted

I would introduce it with a paragraph or two right where committing is
covered the first time. Explain that the empty file list box to the
left contains the changes that will be committed when you press the
commit button, and that the file list box on the right contains the
changes that won't be committed. By clicking on a file name you get to
see the diff to the file, and by clicking on the icon you move it to
the other file list box -- that is, you stage/unstage it.

And now comes the clever part: Introduce the index, by explaining that
it essentially _is_ the left file list box. Explain that git-add is
the command-line equivalent of moving changes to the left box, and
that git-commit without arguments simply commits what's in the index
-- exactly like git-gui's Commit button.

I think it could work. :-)

Definitely, sounds fun.

For the in-tree documentation, maybe I'm just my crusty text-centric
commandline point of view, but I'd rather have the primary explanation
continue to depend only on text and commandline examples, and then add a
note telling people that playing with git-gui may help develop their
intuition for the way the index works.

But I think it'd be interesting to try out the above approach with
screenshots, etc., on a web page someplace.  It might also make a good
visual aid for a talk.

Usually a wiki is a perfect place to start this...

Ciao,
Dscho

On Tue, May 08, 2007 at 06:45:32AM CEST, Linus Torvalds wrote:

Git used explicit index updates from day 1, even before it did the first 
merge. It's simply how I've always worked. I tend to have dirty trees, 
with some random patch in my tree that I do *not* want to commit, because 
it's just a Makefile update for the next version (to remind me - I've 
released kernel versions too many times with an old version number, just 
because I forgot to update the Makefile).

Or other things like that - I have small test-patches in my tree that I 
want to build, but that I don't want to commit, and I end up doing big 
merges and whole patch-application sequences with such a dirty tree 
(obviously if the patch or merge wants to change that file, I then need to 
do something about that dirty state, but it happens surprisingly seldom).

Hmm, does this really work so well for you guys? Because thanks to Mr.
Murphy, in my case, when I have some custom Makefile tweak, I always
need to commit some unrelated changes involving Makefile more often than
usual, and so on; so in general case, file-level changes exclusion
doesn't really work so well for me.

So this use of index seems to me really as a workaround for more
fine-grained change control (in a similar way that rename following
would be a workaround for lack of more fine-grained content moves
tracking). I will have to look into git-gui's hunk-level control and
maybe reimplement it in tig.

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

On Wed, 9 May 2007, Petr Baudis wrote:

On Tue, May 08, 2007 at 06:45:32AM CEST, Linus Torvalds wrote:

quoted

Or other things like that - I have small test-patches in my tree that I 
want to build, but that I don't want to commit, and I end up doing big 
merges and whole patch-application sequences with such a dirty tree 
(obviously if the patch or merge wants to change that file, I then need to 
do something about that dirty state, but it happens surprisingly seldom).

Hmm, does this really work so well for you guys? Because thanks to Mr.
Murphy, in my case, when I have some custom Makefile tweak, I always
need to commit some unrelated changes involving Makefile more often than
usual, and so on; so in general case, file-level changes exclusion
doesn't really work so well for me.

Well, one thing is that I obviously mainly work on a relatively large 
project, and one that has been carefully de-centralized over a long long 
time, so the source code I work with - the kernel - may be more amenable 
to my workflow than most.

For example, we have long long since tried to avoid having central files 
that everybody changes - because it's such a pain to manage, even with 
good automated merging (and even more with central people still using just 
series of patches).

In other words, in well-maintained larger projects, you simply don't see 
those kinds of conflicts very often: people don't work on the same files 
very much. I regularly go for days, and easily merging hundreds of 
thousands of lines of changes, with a dirty tree, and the merges don't 
affect it at all.

And if I happen to hit a dirty file, the pull will just say "cannot 
merge", and I can stash away my changes, and just re-do. So the cost of a 
conflict in a dirty tree is very low when it *does* happen.

So this use of index seems to me really as a workaround for more
fine-grained change control (in a similar way that rename following
would be a workaround for lack of more fine-grained content moves
tracking). I will have to look into git-gui's hunk-level control and
maybe reimplement it in tig.

Many people seem to enjoy per-hunk commits, but I seldom do that. Maybe 
it's just because I'm *so* comfortable with diffs, that when I clean up an 
ugly sequence of commits, what I do is literally:

 - I make sure that my ugly sequence of commits is on some temporary 
   branch, but that the _end_result_ is good and clean (ie I will have 
   tested the end result fairly well, and made sure that there are no 
   debug statements etc crud left).

   I would call this branch something like "target", because the end 
   result of that branch is what I'm looking for - even if the commits in 
   the sequence that gets me there are individually ugly!

 - I just switch back to my starting point (and now I'm usually on 
   "master"), and do

	git diff -R target > diff

   to create a diff of my current tree (which is initially the starting 
   point) to the good result.

 - I actually edit the "diff" file by hand, and edit it down to the part I 
   actually want to commit as the first in the series. And then I just do 
   a "git-apply diff" to actually apply that part to my working tree.

 - I then edit any missing parts in the actual working tree (for example, 
   if there were mixed hunks that I want to get to in later commits, and I 
   edited out above, or that I need to partially undo), to do any 
   finishing touches.

 - I now have a tree I can compile and test, and has the "first part" of 
   the journey towards the final "target" state. If compiling/testing 
   shows that I missed something, I can still fix things, and/or go back 
   to doing another "git diff -R target" to see if I missed something).

 - I commit that first case, and repeat the sequence from step 2 (and 
   at every step, the "diff" file ends up shrinking and shrinking).

The above sounds like it's a complicated sequence, but it really isn't. 
Partly because I just am very comfortable with diffs indeed (probably more 
than most people), but partly because at all times "git diff" works fine 
to see what I've done, and what the diff to "target" is.

And unlike the "simpler" model of committing individual hunks with "git 
add -i" or something like that, my model is actually much superior! It 
means that I can actually test each stage individually, and make sure that 
the intermediate commits are good. It also allows me to edit up places 
where the diff mixes up two different things, and the intermediate result 
needs to be different from the final one.

Do I do this very often? No. Most of the time, the changes are separate 
enough that I can just commit one file at a time, and in fact, I can mix 
and match (ie I can do the above thing in the "big picture", but actually 
end up doing one substep where I do just one "diff and edit" phase, but 
then actually commit that as two things by just committing individual 
files separately when they are obviously independent changes).

But the above is literally what I did for the superproject support and for 
some other things where I want to send out the end result in a nice 
sequence of 5-6 patches, but when I was actually *developing* it I ended 
up making more mistakes, and I started out with 10 patches with some total 
braino's that I had to fix, or cleanups that I didn't do in the right 
sequence.

And I actually mix-and-match other ways of working too. For example, if 
some commit in my otherwise ugly "target" sequence was fine, I'll just 
cherry-pick it instead, and re-order things that way. 

The point of this all is that the "git way" is actually very flexible. You 
can keep the tree dirty and not worry about it, and if you always think 
twice before you do "git commit -a" you won't be committing dirty state 
that you didn't intend to commit by mistake.

Of course, if you get so used to doing "git commit -a" that you just do it 
in your sleep, then the dirty tree model won't work for you, because 
you'll simply start committing stuff you didn't intend to commit when 
you're on auto-pilot. But the way I work, I basically always do

	git diff

to see what's in my tree, and I will only use the "-a" flag when I 
*consciously* think "ok, that's all one thing". 

Btw, what goes hand-in-hand with this workflow is the nice ability to 
specify a subtree. So I'll have a dirty tree with two different 
test-things, but since one of them was a filesystem fix, and the other one 
was in the kernel, rather than give all the paths explicitly, I'd do

	git commit fs/

and it will automatically do the right thing (actually, I often end up 
using the two-stage "git add" + "git commit" thing, because one of the 
more common cases for me is that I'm going to commit a merge that I fixed 
up a conflict in, and then you have to do it that way).

			Linus

On Wed, 9 May 2007 08:52:09 -0700 (PDT), Linus Torvalds wrote:
[Snip good description of rebuilding a branch to meet some "target"
state.]

That's all really good stuff. And as you mentioned you sometimes use
cherry-pick during this rebuilding, one can also use "git add -i" to
help with splitting up an ugly commit that should have been multiple
commit.

For example, a sequence might look like this, (I always use "desired"
where you use target):

	git diff HEAD desired | git apply
	git add -i
	git commit
	git reset --hard
	# test here and commit --amend as needed

And repeat that as needed. It's really no different than your "edit
the diff" approach. It's just using "add -i" instead of a text
editor. But I do admit that the commit;reset;test;--amend sequence
might seem a bit too awkward to some people.

test-things, but since one of them was a filesystem fix, and the other one
was in the kernel, rather than give all the paths explicitly, I'd do

	git commit fs/

and it will automatically do the right thing (actually, I often end up
using the two-stage "git add" + "git commit" thing, because one of the
more common cases for me is that I'm going to commit a merge that I fixed
up a conflict in, and then you have to do it that way).

This reminds me of a confusing semantic issue that came about with the
"new" add. It can be quite natural to commit a single file in one step
with:

	git commit some-file.c

or to do that in two steps with:

	git add some-file.c
	git commit

(which is particularly useful if one wants to add multiple files).

I recently found myself wanting to do a similar thing with a directory
path. I can commit a path with:

	git commit path/

but I don't get anything at all like the same semantics if I do:

	git add path/
	git commit

(since "git add" will recursively add all untracked files under path/).

Now the "recursively add all files" behavior is older, and has been an
essential part of git-add forever. But I found it to be not at all
what I wanted in this case, (where I'm now trained to say "git add" to
stage things into the index).

I don't know of any good fix for the problem now. Maybe I'll just need to
remember to break out that old "git update-index" for a situation like
this, but that sure feels clunky.

-Carl

On 5/9/07, Linus Torvalds [off-list ref] wrote:

 - I just switch back to my starting point (and now I'm usually on
   "master"), and do

        git diff -R target > diff

   to create a diff of my current tree (which is initially the starting
   point) to the good result.

 - I actually edit the "diff" file by hand, and edit it down to the part I
   actually want to commit as the first in the series. And then I just do
   a "git-apply diff" to actually apply that part to my working tree.

 - I then edit any missing parts in the actual working tree (for example,
   if there were mixed hunks that I want to get to in later commits, and I
   edited out above, or that I need to partially undo), to do any
   finishing touches.

 - I now have a tree I can compile and test, and has the "first part" of
   the journey towards the final "target" state. If compiling/testing
   shows that I missed something, I can still fix things, and/or go back
   to doing another "git diff -R target" to see if I missed something).

 - I commit that first case, and repeat the sequence from step 2 (and
   at every step, the "diff" file ends up shrinking and shrinking).

Geez,  this is similar [in nature, not scale] to what I've been doing.
After reading about people "right-clicking on hunks in git-gui",
I was convinced I needed to force myself to do more manipulations
inside git itself.  Hmm...

Maybe, in addition to [or in] the User Manual, git should have some
workflow examples, which have been cribbed from various emails
on this list?

Thanks,
-- 
Dana L. How  danahow@gmail.com  +1 650 804 5991 cell

On Wed, May 09, 2007 at 09:33:40AM -0700, Dana How wrote:

Geez,  this is similar [in nature, not scale] to what I've been doing.
After reading about people "right-clicking on hunks in git-gui",
I was convinced I needed to force myself to do more manipulations
inside git itself.  Hmm...

Maybe, in addition to [or in] the User Manual, git should have some
workflow examples, which have been cribbed from various emails
on this list?

That's something several people have asked for, and I think it's a great
idea--I just haven't personally had much time to get to it.  But I'd
happily take even very rough patches and help get them into shape.

The way I'd thought of doing it was having an "examples" section at the
end of each chapter, with subsections for each individual example; see
the one at the end of the "exploring git history" chapter:

	http://www.kernel.org/pub/software/scm/git/docs/user-manual.html#history-examples

They shouldn't use the material introduced in the associated chapter,
but it's also OK to introduce new commands (with references to the man
pages) when their use in the example is pretty self-explanatory.  (In
fact, this is a great way to introduce more commands and options--git
has so many that it would be tedious to try to be comprehensive, but
they'd fit well in examples.)

The patch-editing stuff discussed above might fit best at the end of
"rewriting history and maintaining patch series".

--b.

On Wed, May 09, 2007 at 07:18:45PM CEST, J. Bruce Fields wrote:

On Wed, May 09, 2007 at 09:33:40AM -0700, Dana How wrote:

quoted

Geez,  this is similar [in nature, not scale] to what I've been doing.
After reading about people "right-clicking on hunks in git-gui",
I was convinced I needed to force myself to do more manipulations
inside git itself.  Hmm...

Maybe, in addition to [or in] the User Manual, git should have some
workflow examples, which have been cribbed from various emails
on this list?

That's something several people have asked for, and I think it's a great
idea--I just haven't personally had much time to get to it.  But I'd
happily take even very rough patches and help get them into shape.

The way I'd thought of doing it was having an "examples" section at the
end of each chapter, with subsections for each individual example; see
the one at the end of the "exploring git history" chapter:

	http://www.kernel.org/pub/software/scm/git/docs/user-manual.html#history-examples

They shouldn't use the material introduced in the associated chapter,
but it's also OK to introduce new commands (with references to the man
pages) when their use in the example is pretty self-explanatory.  (In
fact, this is a great way to introduce more commands and options--git
has so many that it would be tedious to try to be comprehensive, but
they'd fit well in examples.)

The patch-editing stuff discussed above might fit best at the end of
"rewriting history and maintaining patch series".

There is some workflow-related discussion accumulated over years in
Documentation/howto/, some of them also already suffering quite of a
bitrot.  :-(

-- 
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Ever try. Ever fail. No matter. // Try again. Fail again. Fail better.
		-- Samuel Beckett

On Wed, May 09, 2007 at 07:26:22PM +0200, Petr Baudis wrote:

There is some workflow-related discussion accumulated over years in
Documentation/howto/, some of them also already suffering quite of a
bitrot.  :-(

Yup.  I think we should one-by-one update those and suck them into the
manual.  (Patches accepted!)

--b.

On Wed, 9 May 2007, Linus Torvalds wrote:

Many people seem to enjoy per-hunk commits, but I seldom do that. Maybe 
it's just because I'm *so* comfortable with diffs, that when I clean up an 
ugly sequence of commits, what I do is literally:

 - I make sure that my ugly sequence of commits is on some temporary 
   branch, but that the _end_result_ is good and clean (ie I will have 
   tested the end result fairly well, and made sure that there are no 
   debug statements etc crud left).

   I would call this branch something like "target", because the end 
   result of that branch is what I'm looking for - even if the commits in 
   the sequence that gets me there are individually ugly!

 - I just switch back to my starting point (and now I'm usually on 
   "master"), and do

	git diff -R target > diff

   to create a diff of my current tree (which is initially the starting 
   point) to the good result.

 - I actually edit the "diff" file by hand, and edit it down to the part I 
   actually want to commit as the first in the series. And then I just do 
   a "git-apply diff" to actually apply that part to my working tree.

 - I then edit any missing parts in the actual working tree (for example, 
   if there were mixed hunks that I want to get to in later commits, and I 
   edited out above, or that I need to partially undo), to do any 
   finishing touches.

 - I now have a tree I can compile and test, and has the "first part" of 
   the journey towards the final "target" state. If compiling/testing 
   shows that I missed something, I can still fix things, and/or go back 
   to doing another "git diff -R target" to see if I missed something).

 - I commit that first case, and repeat the sequence from step 2 (and 
   at every step, the "diff" file ends up shrinking and shrinking).

The above sounds like it's a complicated sequence, but it really isn't. 
Partly because I just am very comfortable with diffs indeed (probably more 
than most people), but partly because at all times "git diff" works fine 
to see what I've done, and what the diff to "target" is.

It only sounds like a complicated sequence because you didn't write a 
script to do it...

$ git checkout -b clean origin
$ git-refine target
  (edit the patch in the editor that pops up)
$ git-refine
Test changes and commit
$ make test
...
$ git commit
  (write message)
$ git-refine
  (edit the patch, etc)
  ...
$ git commit
$ git-refine
All done.

I actually wrote it years ago, but I couldn't describe my workflow well 
enough, so I didn't submit it. If everybody seems to be doing the same 
thing, I can submit my script...

	-Daniel
*This .sig left intentionally blank*

On Wed, 9 May 2007, Daniel Barkalow wrote:

It only sounds like a complicated sequence because you didn't write a 
script to do it...

Well, I actually think it sounds like a complicated sequence because I 
tried to explain what I do.

The "script" parts don't really end up being any smaller, and not 
scripting it actually means that I can (and often do) things outside of a 
strict scripting environment.

As mentioned, I not only mix it up with "git cherry-pick", but since I 
just use "git diff", I can - and do - things like pick only a certain set 
of files to diff and edit the patch on. 

So it's an iterative process at several levels (the "outer" level is the 
act of actually committing each change, and iterating to the next one, 
while the "inner" level is often a sequence of "git diff" exploration), 
it's not very fixed. 

For example, when I said that I do a 

	git diff -R target > diff

that's not strictly true. The "git diff -R" is useful for comparing the 
current working tree to another commit, but quite often I actually end up 
doing it differently, and doing it as

	git diff ..target file > diff
	.. edit ..
	git apply diff

or, if I don't need the edit (ie just the fact that I limit it to a single 
file is a sufficient "edit" in itself), I might just do

	git checkout target file

instead, which will fetch the whole file from the "target" branch (and 
also update it in the index, which may or may not actually be what I want, 
but that's a different issue).

So the "process" as far as I'm concerned is actually much more fluid than 
necessarily always working with diffs. Git gives you so many ways to do 
things like this, and I'm pretty comfortable with lots of them.

			Linus

Carl Worth wrote:

This reminds me of a confusing semantic issue that came about with the
"new" add. It can be quite natural to commit a single file in one step
with:

      git commit some-file.c

or to do that in two steps with:

      git add some-file.c
      git commit

(which is particularly useful if one wants to add multiple files).

I recently found myself wanting to do a similar thing with a directory
path. I can commit a path with:

      git commit path/

but I don't get anything at all like the same semantics if I do:

      git add path/
      git commit

(since "git add" will recursively add all untracked files under path/).

Now the "recursively add all files" behavior is older, and has been an
essential part of git-add forever. But I found it to be not at all
what I wanted in this case, (where I'm now trained to say "git add" to
stage things into the index).

I don't know of any good fix for the problem now. Maybe I'll just need to
remember to break out that old "git update-index" for a situation like
this, but that sure feels clunky.

In the new version of git I *think* you can use "git add -u path/"

  'git-add' [-n] [-v] [-f] [--interactive | -i] [-u] [--] <file>...

  -u::
        Update all files that git already knows about. This is what
        "git commit -a" does in preparation for making a commit.

(in v1.5.2-rc0, documented in v1.5.2-rc3).
-- 
Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

Johannes Schindelin [off-list ref] writes:

Hi,

On Tue, 8 May 2007, Martin Langhoff wrote:

quoted

Heh. Making the index very visible makes sense when you are merging,

You're saying that the main use of the index is to help merging. I have to 
disagree strongly.

When I have been chasing a bug all over the place, and finally found it, 
my working tree is a mess. Lots of assertions, lots of debugging 
statements, some of them commented out. So, now it is cleanup time, right?

The problem is that more often than not, I broke my fix while cleaning up.

Therefore, I now put all changed files into the index (git add -u), and 
clean up the files one by one, always checking with "git diff" and "git 
diff HEAD" what I still have to do.

Why not simply use a temporary branch for this? They're free, and you
can diff just as easily, if not more. And you don't risk losing it if
you slip with a command.

-- 
David Kågedal

Hi,

On Mon, 14 May 2007, David Kågedal wrote:

Johannes Schindelin [off-list ref] writes:

quoted

Therefore, I now put all changed files into the index (git add -u), 
and clean up the files one by one, always checking with "git diff" and 
"git diff HEAD" what I still have to do.

Why not simply use a temporary branch for this? They're free, and you 
can diff just as easily, if not more. And you don't risk losing it if 
you slip with a command.

Because it is much faster to work with the index?

Ciao,
Dscho