I mean, if you have a testcase and want to see how it works, you can
just try it. merge-ort has been released and is part of git already.
But I'll take a stab...
Your testcase with new folders sounds like testcase 12f from t6423 in
our testsuite (from commit 902c521a35 ("t6423: more involved directory
rename test", 2020-10-15)), a case that works with the `ort` merge
strategy but not with `recursive`. So, the `ort` strategy is likely
to be better for your usecase than `recursive`, but it's hard for me
to definitively state that it'll solve all your usecase(s) since you
didn't fully describe them. This is especially important since you
appear to have discovered a segfault (in the first post); I was not
aware of any circumstances when the old merge machinery would do so.
I'd be interested to know if the same issue repeats with the `ort`
strategy, and, if so, whether you can get a stack trace, or find an
example that reproduces the problem, or provide us with access to your
repository. Also, in your second link, you allude to problems in a
case that does not include new folders, in situations where I'd expect
`recursive` and `ort` to behave the same. But it's hard to know the
expected behavior since you didn't provide that many details about the
setup -- e.g. you didn't cover the contents of any of the files (so
it's hard to know if files are more similar to ones with different
names than what you expected to be matched up), it's unclear if there
are other files involved (e.g. if you have multiple copies of some
library and the rename detection is detecting a different copy than
you expected, or if there are lots of additional files besides those
you are listing in your example and they are being renamed
differently, etc.), whether you have lots of large files also being
renamed (perhaps the VDI files you allude to somewhere?), etc., etc.
If you can provide access to a repository and the commands you
reported and what you expected vs. what you saw, I'd try to find some
time to take a look.
I hope that helps, though I'm not sure I've got enough details to
answer all your questions. I guess if I were to summarize, I'd say:
* the `ort` strategy would be better than `recursive` for you, for
one particular type of issue, but probably behave the same on other
areas
* you seem to have useful testcases that might be different than
ones we have in our testsuite or that I know about
* you haven't provided enough information for me to investigate or
answer any further, though.
You have gotten me really curious about this segfaulting testcase --
whether it's just in the recursive backend or also afflicts merge-ort,
and how to duplicate it.
From: Ariel Man <hidden> Date: 2021-09-30 06:58:03
Hi,
Thanks for the detailed answer, but just to clarify, my main question
is regarding ORT strategy, and not recursive.
Mainly in this case, where ORT didn't behave as I expected:
https://stackoverflow.com/questions/69150777/git-mv-doesnt-work-as-expected-wrong-files-location#:~:text=test3.py%0A%E2%94%82%20%20%20%E2%94%94%E2%94%80%E2%94%80%20dir3-,UPDATE,-I%27m%20adding%20one
As I wrote, this specific case is where new folder and files were
created after the consolidation (and not new files under the existing
folder). As you can see, when only new files were added to the
existing folder, the git ORT recognized it and moved the folder
python1 with its files test1.py and test4.py to the right place
(dir2). But when new folder was created on the target (new_test with
the file test5.py), when I ran the ORT command, the GIT didn't
recognized it and created again the folder python1 on the old path
(dir1-->python1-->new_test-->test5.py).
I believe you can reproduce it yourself on your repository and see the
behaviour.
Ariel
בתאריך יום ה׳, 30 בספט׳ 2021 ב-8:27 מאת Elijah Newren
<newren@gmail.com>:
On Sun, Sep 19, 2021 at 8:12 PM Ariel Man [off-list ref] wrote:
I mean, if you have a testcase and want to see how it works, you can
just try it. merge-ort has been released and is part of git already.
But I'll take a stab...
Your testcase with new folders sounds like testcase 12f from t6423 in
our testsuite (from commit 902c521a35 ("t6423: more involved directory
rename test", 2020-10-15)), a case that works with the `ort` merge
strategy but not with `recursive`. So, the `ort` strategy is likely
to be better for your usecase than `recursive`, but it's hard for me
to definitively state that it'll solve all your usecase(s) since you
didn't fully describe them. This is especially important since you
appear to have discovered a segfault (in the first post); I was not
aware of any circumstances when the old merge machinery would do so.
I'd be interested to know if the same issue repeats with the `ort`
strategy, and, if so, whether you can get a stack trace, or find an
example that reproduces the problem, or provide us with access to your
repository. Also, in your second link, you allude to problems in a
case that does not include new folders, in situations where I'd expect
`recursive` and `ort` to behave the same. But it's hard to know the
expected behavior since you didn't provide that many details about the
setup -- e.g. you didn't cover the contents of any of the files (so
it's hard to know if files are more similar to ones with different
names than what you expected to be matched up), it's unclear if there
are other files involved (e.g. if you have multiple copies of some
library and the rename detection is detecting a different copy than
you expected, or if there are lots of additional files besides those
you are listing in your example and they are being renamed
differently, etc.), whether you have lots of large files also being
renamed (perhaps the VDI files you allude to somewhere?), etc., etc.
If you can provide access to a repository and the commands you
reported and what you expected vs. what you saw, I'd try to find some
time to take a look.
I hope that helps, though I'm not sure I've got enough details to
answer all your questions. I guess if I were to summarize, I'd say:
* the `ort` strategy would be better than `recursive` for you, for
one particular type of issue, but probably behave the same on other
areas
* you seem to have useful testcases that might be different than
ones we have in our testsuite or that I know about
* you haven't provided enough information for me to investigate or
answer any further, though.
You have gotten me really curious about this segfaulting testcase --
whether it's just in the recursive backend or also afflicts merge-ort,
and how to duplicate it.
Hi,
Please do not top-post on this mailing list.
On Wed, Sep 29, 2021 at 11:58 PM Ariel Man [off-list ref] wrote:
Hi,
Thanks for the detailed answer, but just to clarify, my main question
is regarding ORT strategy, and not recursive.
Since you simply linked to three posts in your original email, and the
first showed a segfault, this wasn't at all clear.
Mainly in this case, where ORT didn't behave as I expected:
https://stackoverflow.com/questions/69150777/git-mv-doesnt-work-as-expected-wrong-files-location#:~:text=test3.py%0A%E2%94%82%20%20%20%E2%94%94%E2%94%80%E2%94%80%20dir3-,UPDATE,-I%27m%20adding%20one
As I wrote, this specific case is where new folder and files were
created after the consolidation (and not new files under the existing
folder). As you can see, when only new files were added to the
existing folder, the git ORT recognized it and moved the folder
python1 with its files test1.py and test4.py to the right place
(dir2). But when new folder was created on the target (new_test with
the file test5.py), when I ran the ORT command, the GIT didn't
recognized it and created again the folder python1 on the old path
(dir1-->python1-->new_test-->test5.py).
Yes, I read through that and pointed out your examples weren't fully
specified. You haven't stated what else exists in all these
directories, what kind of contents files had, if there were big
binaries, etc.
I believe you can reproduce it yourself on your repository and see the
behaviour.
Nope, I can't. I went through the trouble of setting up the testcases
as close as I could from your diagrams. When using the recursive
strategy, I do see that test4.py and test5.py are left behind in dir1,
but when using the ort strategy, it behaves exactly as you said you
wanted (with *both* test4.py and test5.py appearing somewhere under
dir2 rather than dir1).
Here's the output from my script showing all the files that exist in
each of the relevant commits:
=== Original ===
dir1/otherfile
dir1/python1/test1.py
dir2/otherfile
dir2/python2/test2.py
dir3/otherfile
dir3/python3/test3.py
=== Consolidated ===
dir1/otherfile
dir2/otherfile
dir2/python1/test1.py
dir2/python2/test2.py
dir2/python3/test3.py
dir3/otherfile
=== New files ===
dir1/otherfile
dir1/python1/new_test/test5.py
dir1/python1/test1.py
dir1/python1/test4.py
dir2/otherfile
dir2/python2/test2.py
dir3/otherfile
dir3/python3/test3.py
=== Merged result ===
dir1/otherfile
dir2/otherfile
dir2/python1/new_test/test5.py
dir2/python1/test1.py
dir2/python1/test4.py
dir2/python2/test2.py
dir2/python3/test3.py
dir3/otherfile
and here's the script I used to run this testcase and generate the above output:
#!/bin/bash
git init -b main nukeme
cd nukeme
echo '''
│ └── dir1
│ │ └── python1
│ │ │ └── test1.py
│ └── dir2
│ │ └── python2
│ │ │ └── test2.py
│ └── dir3
│ │ └── python3
│ │ │ └── test3.py
''' >/dev/null
for i in $(seq 1 3); do
mkdir -p dir$i/python$i
echo $i >dir$i/python$i/test$i.py
>dir$i/otherfile
done
git add .
git commit -qm "Initial"
git branch consolidated
git branch newfiles
git checkout consolidated
echo '''
│ └── dir1
│ └── dir2
│ │ └── python1
│ │ │ └── test1.py
│ │ └── python2
│ │ │ └── test2.py
│ │ └── python3
│ │ │ └── test3.py
│ └── dir3
''' >/dev/null
git mv dir1/python1 dir2/
git mv dir3/python3 dir2/
git commit -qm consolidated
git checkout newfiles
echo 4 >dir1/python1/test4.py
git add dir1/python1/test4.py
mkdir dir1/python1/new_test
echo 5 >dir1/python1/new_test/test5.py
git add dir1/python1/new_test/test5.py
git commit -qm newfiles
git checkout -qb merged consolidated
git merge --no-edit -s ort newfiles
echo
echo "=== Original ==="
git ls-tree -r --name-only main
echo
echo "=== Consolidated ==="
git ls-tree -r --name-only consolidated
echo
echo "=== New files ==="
git ls-tree -r --name-only newfiles
echo
echo "=== Merged result ==="
git ls-tree -r --name-only merged