recovering from "unordered stage entries in index" error

	I've just upgraded my git from 2.0.5 to 2.3.6, and I'm now unable to run 'git svn fetch' in one of my repositories:

$ git svn fetch
fatal: unordered stage entries in index
write-tree: command returned error: 128

	'git status' shows a few untracked files but is otherwise clean.

	It looks like this check was introduced in 15999d0be8179fb7a2e6eafb931d25ed65df50aa, with the summary "read_index_from(): catch out of order entries when reading an index file" (first appearing in 2.2.0).

	Mailing list discussion looked like it implicated third-party tools.  I don't recall running any other tools on this repo; it doesn't do much day-to-day other than a long series of 'git svn fetch'es.  (But it's been around for a couple of years, so who knows.)

	At any rate, what can I do to recover from this situation?  I tried to locate a path with multiple index entries like this, but got no results:

$ git ls-files -s | cut -f 2-100 | sort | uniq -c | grep -v '^[ \t]*1 '

	(I originally posted on SO at http://stackoverflow.com/questions/30264826/; I'll update that with any solutions that come up here, to ease future googling.)

On Tue, May 19, 2015 at 6:48 AM, McHenry, Matt
[off-list ref] wrote:

        I've just upgraded my git from 2.0.5 to 2.3.6, and I'm now unable to run 'git svn fetch' in one of my repositories:

$ git svn fetch
fatal: unordered stage entries in index
write-tree: command returned error: 128

Are you by any chance using the split-index mode (which I think is
broken in 2.3.6)?

	I had never heard of that.  Google led me to the man page for 'git update-index --split-index', and after reading it, I think this demonstrates that I'm not using that feature:

$ ls .git/*index*
.git/index

-----Original Message-----
From: jch2355@gmail.com [mailto:jch2355@gmail.com] On Behalf Of Junio C
Hamano
Sent: Tuesday, May 19, 2015 11:20
To: McHenry, Matt
Cc: git@vger.kernel.org
Subject: Re: recovering from "unordered stage entries in index" error

On Tue, May 19, 2015 at 6:48 AM, McHenry, Matt
[off-list ref] wrote:

quoted

        I've just upgraded my git from 2.0.5 to 2.3.6, and I'm now

unable to run 'git svn fetch' in one of my repositories:

quoted

$ git svn fetch
fatal: unordered stage entries in index
write-tree: command returned error: 128

Are you by any chance using the split-index mode (which I think is
broken in 2.3.6)?

On Tue, May 19, 2015 at 8:48 PM, McHenry, Matt
[off-list ref] wrote:

        I've just upgraded my git from 2.0.5 to 2.3.6, and I'm now unable to run 'git svn fetch' in one of my repositories:

$ git svn fetch
fatal: unordered stage entries in index

This message can be improved to show what entries have this problem.
But then I don't see any way to recover the index manually. ls-files
will die too. Perhaps we should be gentle in this case: show warnings
instead of aborting the program and internally reorder the index. I
think, unless you have multiple entries with the same stage, the
recovered index should run well. The broken index could be renamed to
index.broken or something for later analysis, or we forbid writing the
reordered index to disk.

Hmm?

write-tree: command returned error: 128

        'git status' shows a few untracked files but is otherwise clean.

        It looks like this check was introduced in 15999d0be8179fb7a2e6eafb931d25ed65df50aa, with the summary "read_index_from(): catch out of order entries when reading an index file" (first appearing in 2.2.0).

        Mailing list discussion looked like it implicated third-party tools.  I don't recall running any other tools on this repo; it doesn't do much day-to-day other than a long series of 'git svn fetch'es.  (But it's been around for a couple of years, so who knows.)

        At any rate, what can I do to recover from this situation?  I tried to locate a path with multiple index entries like this, but got no results:

$ git ls-files -s | cut -f 2-100 | sort | uniq -c | grep -v '^[ \t]*1 '

        (I originally posted on SO at http://stackoverflow.com/questions/30264826/; I'll update that with any solutions that come up here, to ease future googling.)
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Duy

This message can be improved to show what entries have this problem.

	Yes, that would definitely be a start.  :)

But then I don't see any way to recover the index manually. ls-files
will die too. Perhaps we should be gentle in this case: show warnings

	Actually, ls-files succeeds on my broken index:

$ git ls-files > /dev/null
$ echo $?
0

	Could I do something with 'git read-tree' to force creation of a new valid index?  I guess 'git clone' would work too, except that I have 'git svn' metadata that I'd need to preserve.

instead of aborting the program and internally reorder the index. I
think, unless you have multiple entries with the same stage, the
recovered index should run well. The broken index could be renamed to
index.broken or something for later analysis, or we forbid writing the
reordered index to disk.

Hmm?

quoted

write-tree: command returned error: 128

        'git status' shows a few untracked files but is otherwise clean.

        It looks like this check was introduced in

15999d0be8179fb7a2e6eafb931d25ed65df50aa, with the summary
"read_index_from(): catch out of order entries when reading an index file"
(first appearing in 2.2.0).

quoted

        Mailing list discussion looked like it implicated third-party

tools.  I don't recall running any other tools on this repo; it doesn't do
much day-to-day other than a long series of 'git svn fetch'es.  (But it's
been around for a couple of years, so who knows.)

quoted

        At any rate, what can I do to recover from this situation?  I

tried to locate a path with multiple index entries like this, but got no
results:

quoted

$ git ls-files -s | cut -f 2-100 | sort | uniq -c | grep -v '^[ \t]*1 '

        (I originally posted on SO at

http://stackoverflow.com/questions/30264826/; I'll update that with any
solutions that come up here, to ease future googling.)

quoted

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
Duy

The message "unordered stage entries in index" comes only when
two adjacent entries in the index are in a wrong order, e.g. "test0"
should come before "test1" but somehow the index records them
in the other way around. Doing something like this:

$ git ls-files >Q
$ LANG=C LC_ALL=C sort Q >R
$ diff Q R

may tell you which entries are wrong, even though it wouldn't show
who made them wrong.

(pardon top-posting, overlong lines and typos; sent from GMail web UI)

On Tue, May 19, 2015 at 6:48 AM, McHenry, Matt
[off-list ref] wrote:

        I've just upgraded my git from 2.0.5 to 2.3.6, and I'm now unable to run 'git svn fetch' in one of my repositories:

$ git svn fetch
fatal: unordered stage entries in index
write-tree: command returned error: 128

        'git status' shows a few untracked files but is otherwise clean.

        It looks like this check was introduced in 15999d0be8179fb7a2e6eafb931d25ed65df50aa, with the summary "read_index_from(): catch out of order entries when reading an index file" (first appearing in 2.2.0).

        Mailing list discussion looked like it implicated third-party tools.  I don't recall running any other tools on this repo; it doesn't do much day-to-day other than a long series of 'git svn fetch'es.  (But it's been around for a couple of years, so who knows.)

        At any rate, what can I do to recover from this situation?  I tried to locate a path with multiple index entries like this, but got no results:

$ git ls-files -s | cut -f 2-100 | sort | uniq -c | grep -v '^[ \t]*1 '

        (I originally posted on SO at http://stackoverflow.com/questions/30264826/; I'll update that with any solutions that come up here, to ease future googling.)
--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

	Yes, that does turn up some interesting stuff.  It looks like the repository contains some paths with non-ASCII characters, for example this one has some en-dashes (U+2013) in its name:

$ svn ls -R svn://dev/trunk/curriculum/Fluency | grep Ninja__Beta
Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 – Ninja/FT3 – Ninja__Beta.zip

$ svn ls -R svn://dev/trunk/curriculum/Fluency | grep Ninja__Beta | od -cx
0000000   H   u   r   i   x       w   o   r   k   /   s   o   u   r   c
           7548    6972    2078    6f77    6b72    732f    756f    6372
0000020   e       f   r   o   m       M   a   y       2   0   1   4   /
           2065    7266    6d6f    4d20    7961    3220    3130    2f34
0000040   F   o   r   _   A   n   e   s   h   /   0   6       D   e   l
           6f46    5f72    6e41    7365    2f68    3630    4420    6c65
0000060   i   v   e   r   a   b   l   e   s   /   P   h   a   s   e
           7669    7265    6261    656c    2f73    6850    7361    2065
0000100   2   /   F   T   3     342 200 223       N   i   n   j   a   /
           2f32    5446    2033    80e2    2093    694e    6a6e    2f61
0000120   F   T   3     342 200 223       N   i   n   j   a   _   _   B
           5446    2033    80e2    2093    694e    6a6e    5f61    425f
0000140   e   t   a   .   z   i   p  \n
           7465    2e61    697a    0a70
0000150

	In the output of 'git ls-files', those paths appear quoted (there are almost 100 of them):

$ git ls-files | grep Ninja__Beta
"curriculum/Fluency/Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 \342\200\223 Ninja/FT3 \342\200\223 Ninja__Beta.zip"

$ git ls-files | grep ^\" | wc -l
89


	In the diff you suggested, 'sort' puts those paths at the absolute top of the list, while plain old ls-files puts them inline with the rest of the contents of the curriculum/ subdir:

$ grep -n Ninja__Beta Q R
Q:36109:"curriculum/Fluency/Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 \342\200\223 Ninja/FT3 \342\200\223 Ninja__Beta.zip"
R:89:"curriculum/Fluency/Hurix work/source from May 2014/For_Anesh/06 Deliverables/Phase 2/FT3 \342\200\223 Ninja/FT3 \342\200\223 Ninja__Beta.zip"

	Also, I have the curriculum/Fluency/ directory marked as sparse-checkout:

$ cat .git/info/sparse-checkout
/*
!/curriculum/Fluency/
!/curriculum/Problems/lisp/
!/curriculum/Problems/lisp_es/
!/curriculum/Problems/sdk/Geometry/
!/curriculum/Problems/sdk_es/Geometry/
!/curriculum/Problems/sdk/Test-Questions/
!/curriculum/Problems/sdk_es/Test-Questions/
!/curriculum/Problems/sdk/Grammar/


	However, I tried to construct a test case that would reproduce this with a simple SVN repo containing a file created by 'touch "make-git-svn-$(echo -e '\u201c')unhappy$(echo -e '\u201d')"', but could not get it to fail.  So there may be something more subtle going on here ...

-----Original Message-----
From: jch2355@gmail.com [mailto:jch2355@gmail.com] On Behalf Of Junio C
Hamano
Sent: Friday, May 22, 2015 15:25
To: McHenry, Matt
Cc: git@vger.kernel.org
Subject: Re: recovering from "unordered stage entries in index" error

The message "unordered stage entries in index" comes only when
two adjacent entries in the index are in a wrong order, e.g. "test0"
should come before "test1" but somehow the index records them
in the other way around. Doing something like this:

$ git ls-files >Q
$ LANG=C LC_ALL=C sort Q >R
$ diff Q R

may tell you which entries are wrong, even though it wouldn't show
who made them wrong.

(pardon top-posting, overlong lines and typos; sent from GMail web UI)

On Tue, May 19, 2015 at 6:48 AM, McHenry, Matt
[off-list ref] wrote:

quoted

        I've just upgraded my git from 2.0.5 to 2.3.6, and I'm now

unable to run 'git svn fetch' in one of my repositories:

quoted

$ git svn fetch
fatal: unordered stage entries in index
write-tree: command returned error: 128

        'git status' shows a few untracked files but is otherwise clean.

        It looks like this check was introduced in

15999d0be8179fb7a2e6eafb931d25ed65df50aa, with the summary
"read_index_from(): catch out of order entries when reading an index file"
(first appearing in 2.2.0).

quoted

        Mailing list discussion looked like it implicated third-party

tools.  I don't recall running any other tools on this repo; it doesn't do
much day-to-day other than a long series of 'git svn fetch'es.  (But it's
been around for a couple of years, so who knows.)

quoted

        At any rate, what can I do to recover from this situation?  I

tried to locate a path with multiple index entries like this, but got no
results:

quoted

$ git ls-files -s | cut -f 2-100 | sort | uniq -c | grep -v '^[ \t]*1 '

        (I originally posted on SO at

http://stackoverflow.com/questions/30264826/; I'll update that with any
solutions that come up here, to ease future googling.)

quoted

--
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html