Re: Round-tripping fast-export/import changes commit hashes

7 messages, 3 authors, 2023-01-13 · open the first message on its own page

Re: Round-tripping fast-export/import changes commit hashes

From: Junio C Hamano <hidden>
Date: 2021-03-04 01:09:26

Johannes Sixt [off-list ref] writes:
Am 02.03.21 um 22:52 schrieb anatoly techtonik:
quoted
For my use case, where I just need to attach another branch in
time without altering original commits in any way, `reposurgeon`
can not be used.
What do you mean by "attach another branch in time"? Because if you
really do not want to alter original commits in any way, perhaps you
only want `git fetch /the/other/repository master:the-other-one-s-master`?
Yeah, I had the same impression.  If a bit-for-bit identical copy of
the original history is needed, then fetching from the original
repository (either directly or via a bundle) would be a much simpler
and performant way.

Thanks.

Re: Round-tripping fast-export/import changes commit hashes

From: anatoly techtonik <hidden>
Date: 2021-08-09 15:45:44

On Thu, Mar 4, 2021 at 3:56 AM Junio C Hamano [off-list ref] wrote:
Johannes Sixt [off-list ref] writes:
quoted
Am 02.03.21 um 22:52 schrieb anatoly techtonik:
quoted
For my use case, where I just need to attach another branch in
time without altering original commits in any way, `reposurgeon`
can not be used.
What do you mean by "attach another branch in time"? Because if you
really do not want to alter original commits in any way, perhaps you
only want `git fetch /the/other/repository master:the-other-one-s-master`?
Yeah, I had the same impression.  If a bit-for-bit identical copy of
the original history is needed, then fetching from the original
repository (either directly or via a bundle) would be a much simpler
and performant way.
The goal is to have an editable stream, which, if left without edits, would
be bit-by-bit identical, so that external tools like `reposurgeon` could
operate on that stream and be audited.

Right now, because the repository
https://github.com/simons-public/protonfixes contains a signed commit
right from the start, the simple fast-export and fast-import with git itself
fails the check.

I understand that patching `git` to add `--complete` to fast-import is
realistically beyond my coding abilities, and my only option is to parse
the binary stream produced by `git cat-file --batch`, which I also won't
be able to do without specification.

P.S. I am resurrecting the old thread, because my problem with editing
the history of the repository with an external tool still can not be solved.
-- 
anatoly t.

Re: Round-tripping fast-export/import changes commit hashes

From: Elijah Newren <hidden>
Date: 2021-08-09 18:15:59

On Mon, Aug 9, 2021 at 8:45 AM anatoly techtonik [off-list ref] wrote:
On Thu, Mar 4, 2021 at 3:56 AM Junio C Hamano [off-list ref] wrote:
quoted
Johannes Sixt [off-list ref] writes:
quoted
Am 02.03.21 um 22:52 schrieb anatoly techtonik:
quoted
For my use case, where I just need to attach another branch in
time without altering original commits in any way, `reposurgeon`
can not be used.
What do you mean by "attach another branch in time"? Because if you
really do not want to alter original commits in any way, perhaps you
only want `git fetch /the/other/repository master:the-other-one-s-master`?
Yeah, I had the same impression.  If a bit-for-bit identical copy of
the original history is needed, then fetching from the original
repository (either directly or via a bundle) would be a much simpler
and performant way.
The goal is to have an editable stream, which, if left without edits, would
be bit-by-bit identical, so that external tools like `reposurgeon` could
operate on that stream and be audited.
There were some patches proposed some months back[1] to make
fast-import allow importing signed commits...except that they
unconditionally kept the signatures and didn't do any validation,
which would have resulted in invalid signatures if any edits happened.
I suggested adding signature verification (which would allow options
like erroring out if they didn't match, or dropping signatures when
they didn't match but keeping them otherwise).  That'd help usecases
like yours.  The author wasn't interested in implementing that
suggestion (and it's a low priority for me that I may never get around
to).  The series also wasn't pushed through and eventually was
dropped.

However, that wouldn't fully solve your stated goal.  As already
mentioned earlier in this thread, I don't think your stated goal is
realistic; the only complete bit-for-bit identical representation of
the repository is the original binary format.

Your stated goal here, however, isn't required for solving the usecase
you present.

[1] https://lore.kernel.org/git/20210430232537.1131641-1-lukeshu@lukeshu.com/
Right now, because the repository
https://github.com/simons-public/protonfixes contains a signed commit
right from the start, the simple fast-export and fast-import with git itself
fails the check.
Yes, and I mentioned several other reasons why a round-trip from
fast-export through fast-import cannot be relied upon to preserve
object hashes.
I understand that patching `git` to add `--complete` to fast-import is
realistically beyond my coding abilities, and my only option is to parse
It's more patching than that which would be required:
(1) It'd be both fast-export and fast-import that would need patching,
not just fast-import.
(2) --complete is a bit of a misnomer too, because it's not just
get-all-the-data, it's keep-the-data-in-the-original-format.  If
objects had modes of 040000 instead of 40000, despite meaning the same
thing, you'd have to prevent canonicalization and store them as the
original recorded value or you'd get a different hash.  Ditto for
commit messages with extra data after a NUL byte, and a variety of
other possible issues.
(3) fast-export works by looking for the relevant bits it knows how to
export.  You'd have to redesign it to fully parse every bit of data in
each object it looks at, throw errors if it didn't recognize any, and
make sure it exports all the bits.  That might be difficult since it's
hard to know how to future proof it.  How do you guarantee you've
printed every field in a commit struct, when that struct might gain
new fields in the future?  (This is especially challenging since
fast-export/fast-import might not be considered core tools, or at
least don't get as much attention as the "truly core" parts of git;
see https://lore.kernel.org/git/xmqq36mxdnpz.fsf@gitster-ct.c.googlers.com/)
the binary stream produced by `git cat-file --batch`, which I also won't
be able to do without specification.
The specification is already available in the manual.  Just run `git
cat-file --help` to see it.  Let me quote part of it for you:

       For example, --batch without a custom format would produce:

           <sha1> SP <type> SP <size> LF
           <contents> LF
P.S. I am resurrecting the old thread, because my problem with editing
the history of the repository with an external tool still can not be solved.
Sure it can, just use fast-export's --reference-excluded-parents
option and don't export commits you know you won't need to change.

Or, if for some reason you are really set on exporting everything and
then editing, then go ahead and create the full fast-export output,
including with all your edits, and then post-process it manually
before feeding to fast-import.  In particular, in the post-processing
step find the commits that were problematic that you know won't be
modified, such as your signed commit.  Then go edit that fast-export
dump and (a) remove the dump of the no-longer-signed signed commit
(because you don't want it), and (b) replace any references to the
no-longer-signed-commit (e.g. "from :12") to instead use the hash of
the actual original signed commit (e.g. "from
d3d24b63446c7d06586eaa51764ff0c619113f09").  If you do that, then git
fast-import will just build the new commits on the existing signed
commit instead of on some new commit that is missing the signature.
Technically, you can even skip step (a), as all it will do is produce
an extra commit in your repository that isn't used and thus will be
garbage collected later.

Re: Round-tripping fast-export/import changes commit hashes

From: anatoly techtonik <hidden>
Date: 2021-08-10 15:52:01

On Mon, Aug 9, 2021 at 9:15 PM Elijah Newren [off-list ref] wrote:
The author wasn't interested in implementing that
suggestion (and it's a low priority for me that I may never get around
to).  The series also wasn't pushed through and eventually was
dropped.
What it takes to validate the commit signature? Isn't it the same as
validating commit tag? Is it possible to merge at least the `--fast-export`
part? The effect of roundtrip would be the same, but at least external
tools would be able to detect signed commits and warn users.
[1] https://lore.kernel.org/git/20210430232537.1131641-1-lukeshu@lukeshu.com/
Yes, and I mentioned several other reasons why a round-trip from
fast-export through fast-import cannot be relied upon to preserve
object hashes.
Yes, I understand that. What would be the recommended way to detect
which commits would change as a result of the round-trip? It will then
be possible to warn users in `reposurgeon` `lint` command.
(3) fast-export works by looking for the relevant bits it knows how to
export.  You'd have to redesign it to fully parse every bit of data in
each object it looks at, throw errors if it didn't recognize any, and
make sure it exports all the bits.  That might be difficult since it's
hard to know how to future proof it.  How do you guarantee you've
printed every field in a commit struct, when that struct might gain
new fields in the future?  (This is especially challenging since
fast-export/fast-import might not be considered core tools, or at
least don't get as much attention as the "truly core" parts of git;
see https://lore.kernel.org/git/xmqq36mxdnpz.fsf@gitster-ct.c.googlers.com/)
Looks like the only way to make it forward compatible is to introduce
some kind of versioning and a validation schema like protobuf. Otherwise
writing an importer and exporter for each and every thing that may
encounter in a git stream may be unrealistic, yes.
quoted
P.S. I am resurrecting the old thread, because my problem with editing
the history of the repository with an external tool still can not be solved.
Sure it can, just use fast-export's --reference-excluded-parents
option and don't export commits you know you won't need to change.
How does `--reference-excluded-parents` help to read signed commits?

`reposurgeon` needs all commits to select those that are needed by
different criteria. It is hard to tell which commits are not important without
reading and processing them first.
Or, if for some reason you are really set on exporting everything and
then editing, then go ahead and create the full fast-export output,
including with all your edits, and then post-process it manually
before feeding to fast-import.  In particular, in the post-processing
step find the commits that were problematic that you know won't be
modified, such as your signed commit.  Then go edit that fast-export
dump and (a) remove the dump of the no-longer-signed signed commit
(because you don't want it), and (b) replace any references to the
no-longer-signed-commit (e.g. "from :12") to instead use the hash of
the actual original signed commit (e.g. "from
d3d24b63446c7d06586eaa51764ff0c619113f09").  If you do that, then git
fast-import will just build the new commits on the existing signed
commit instead of on some new commit that is missing the signature.
Technically, you can even skip step (a), as all it will do is produce
an extra commit in your repository that isn't used and thus will be
garbage collected later.
The problem is to detect problematic signed commits, because as I
understand `fast-export` doesn't give any signs if commits were signed
before the export.
-- 
anatoly t.

Re: Round-tripping fast-export/import changes commit hashes

From: Elijah Newren <hidden>
Date: 2021-08-10 18:18:08

On Tue, Aug 10, 2021 at 8:51 AM anatoly techtonik [off-list ref] wrote:
On Mon, Aug 9, 2021 at 9:15 PM Elijah Newren [off-list ref] wrote:
quoted
The author wasn't interested in implementing that
suggestion (and it's a low priority for me that I may never get around
to).  The series also wasn't pushed through and eventually was
dropped.
What it takes to validate the commit signature?
I'm not familiar with any of the gpg libraries, and don't even have an
active gpg key.  So, I don't know.  Some quick grepping shows that we
have gpg-interface.[ch], so we have some functions we can apparently
call.
Isn't it the same as validating commit tag?
gpg signatures of tags are somewhat different than gpg signatures of commits:

* gpg signatures for tags are simply part of the annotated tag message
* gpg signatures for commits are stored in a separate commit header,
not just as extra text at the end of the commit message

This gpg signature handling for tags means that fast-import isn't even
aware of whether the tag is signed; it simply sees a commit message
and records it.  fast-export also would have been unaware and just
exported them as-is if someone hadn't written some special parsing for
it.  fast-import would need to do similar special parsing to become
aware of whether the tags are signed or not.  For now, fast-import
just keeps any tag messages as-is, and thus potentially writes invalid
tag signatures.  (The only way people have to control this is at the
fast-export side with the --signed-tags flag, which gives you the
choices of abort, strip, or keep the signatures even though they'll
likely be wrong.)  If fast-import were to gain knowledge of tag
signatures and an ability to validate them, it could offer smarter
options like keep-if-valid-and-discard-otherwise.

In contrast, the fact that gpg signatures for commits have to be
recorded as a separate commit header means they cannot be recorded in
fast-import without additional code changes.  And both the fast-export
and fast-import sides have to be made aware of and specially handle
the commit signatures for them to even get propagated, let alone
validated.
Is it possible to merge at least the `--fast-export`
part? The effect of roundtrip would be the same, but at least external
tools would be able to detect signed commits and warn users.
The fact that it wasn't merged suggests there was some issue raised in
feedback that wasn't addressed.  I don't remember if that was the case
or not, but someone would have to find out, address any remaining
issues pointed out by feedback, and champion it through.

Personally, I don't like shoving a half solution through and think
there needs to be validation on the fast-import side added at the same
time, but others may disagree with me.  I have plenty of other
projects to work on, though, so whoever does the work will more likely
be the ones to decide.
quoted
[1] https://lore.kernel.org/git/20210430232537.1131641-1-lukeshu@lukeshu.com/
quoted
Yes, and I mentioned several other reasons why a round-trip from
fast-export through fast-import cannot be relied upon to preserve
object hashes.
Yes, I understand that. What would be the recommended way to detect
which commits would change as a result of the round-trip? It will then
be possible to warn users in `reposurgeon` `lint` command.
There is no function or command that would check that kind of thing
short of doing the round-trip.  I provided a list of reasons IDs could
change as a starting point in case anyone wanted to try to write a
function or command that could check, and to point out that it is a
long list and might grow in the future.

I think practically, if you're doing a one-shot export (as I
originally assumed from your email), that you'd find out and then just
manually fix things up by hand.  If your goal is writing or changing a
general purpose filtering tool, then I'd suggest instead using the
alternate technique I outlined in the other thread you started at [2].

[2] https://lore.kernel.org/git/CABPp-BH4dcsW52immJpTjgY5LjaVfKrY9MaUOnKT3byi2tBPpg@mail.gmail.com/
quoted
(3) fast-export works by looking for the relevant bits it knows how to
export.  You'd have to redesign it to fully parse every bit of data in
each object it looks at, throw errors if it didn't recognize any, and
make sure it exports all the bits.  That might be difficult since it's
hard to know how to future proof it.  How do you guarantee you've
printed every field in a commit struct, when that struct might gain
new fields in the future?  (This is especially challenging since
fast-export/fast-import might not be considered core tools, or at
least don't get as much attention as the "truly core" parts of git;
see https://lore.kernel.org/git/xmqq36mxdnpz.fsf@gitster-ct.c.googlers.com/)
Looks like the only way to make it forward compatible is to introduce
some kind of versioning and a validation schema like protobuf. Otherwise
writing an importer and exporter for each and every thing that may
encounter in a git stream may be unrealistic, yes.
quoted
quoted
P.S. I am resurrecting the old thread, because my problem with editing
the history of the repository with an external tool still can not be solved.
Sure it can, just use fast-export's --reference-excluded-parents
option and don't export commits you know you won't need to change.
How does `--reference-excluded-parents` help to read signed commits?
It doesn't.  I was assuming you were doing a one shot export, namely
of the repository you linked to,
https://github.com/simons-public/protonfixes, and that you already
knew which commits were not going to be changed (because you pointed
them out in your email to the list) -- and in fact that it was only a
single commit affected, as you mentioned.

Armed with that knowledge, you could just export the parts of the
repository AFTER that commit, and use --reference-excluded-parents to
make sure the fast-export stream built upon them rather than squashing
all changes up to that point into the first commit in the stream.
`reposurgeon` needs all commits to select those that are needed by
different criteria. It is hard to tell which commits are not important without
reading and processing them first.
Right, so you aren't trying to just handle this one repository, but
modify/create a general purpose tool that does so.  See my response in
the other thread you started, again at [2] above.
quoted
Or, if for some reason you are really set on exporting everything and
then editing, then go ahead and create the full fast-export output,
including with all your edits, and then post-process it manually
before feeding to fast-import.  In particular, in the post-processing
step find the commits that were problematic that you know won't be
modified, such as your signed commit.  Then go edit that fast-export
dump and (a) remove the dump of the no-longer-signed signed commit
(because you don't want it), and (b) replace any references to the
no-longer-signed-commit (e.g. "from :12") to instead use the hash of
the actual original signed commit (e.g. "from
d3d24b63446c7d06586eaa51764ff0c619113f09").  If you do that, then git
fast-import will just build the new commits on the existing signed
commit instead of on some new commit that is missing the signature.
Technically, you can even skip step (a), as all it will do is produce
an extra commit in your repository that isn't used and thus will be
garbage collected later.
The problem is to detect problematic signed commits, because as I
understand `fast-export` doesn't give any signs if commits were signed
before the export.
Signed commits is just one issue, and you'll have to add special code
to handle a bunch of other special cases if you go down this route.
I'd rephrase the problem.  You want to know when _your tool_ (e.g.
reposurgeon since you refer to it multiple times; I'm guessing you're
contributing to it?) has not modified a commit or any of its
ancestors, and when it hasn't, then _your tool_ should remove that
commit from the fast-export stream and replace any references to it by
the original commit's object id.  I outlined how to do this in [2],
referenced above, making use of the --show-original-ids flag to
fast-export.  If you do that, then for any commits which you haven't
modified (including not modifying any of its ancestors), then you'll
keep the same commits as-is with no stripping of gpg-signatures or
canonicalization of objects, so that you'll have the exact same commit
IDs.  Further, you can do this today, without any changes to git
fast-export or git fast-import.

Re: Round-tripping fast-export/import changes commit hashes

From: anatoly techtonik <hidden>
Date: 2022-12-11 18:30:33

On Tue, Aug 10, 2021 at 8:58 PM Elijah Newren [off-list ref] wrote:
On Tue, Aug 10, 2021 at 8:51 AM anatoly techtonik [off-list ref] wrote:
quoted
On Mon, Aug 9, 2021 at 9:15 PM Elijah Newren [off-list ref] wrote:
quoted
[2] https://lore.kernel.org/git/CABPp-BH4dcsW52immJpTjgY5LjaVfKrY9MaUOnKT3byi2tBPpg@mail.gmail.com/

Signed commits is just one issue, and you'll have to add special code
to handle a bunch of other special cases if you go down this route.
I'd rephrase the problem.  You want to know when _your tool_ (e.g.
reposurgeon since you refer to it multiple times; I'm guessing you're
contributing to it?) has not modified a commit or any of its
ancestors, and when it hasn't, then _your tool_ should remove that
commit from the fast-export stream and replace any references to it by
the original commit's object id.  I outlined how to do this in [2],
referenced above, making use of the --show-original-ids flag to
fast-export.  If you do that, then for any commits which you haven't
modified (including not modifying any of its ancestors), then you'll
keep the same commits as-is with no stripping of gpg-signatures or
canonicalization of objects, so that you'll have the exact same commit
IDs.  Further, you can do this today, without any changes to git
fast-export or git fast-import.
Took me a while to process the reply. Let's recap.

I want to make a roundtrip export/import of
https://github.com/simons-public/protonfixes which should get exactly
the same repository.

# --- fast-export to exported.txt
git clone https://github.com/simons-public/protonfixes
git -C protonfixes fast-export --all > exported.txt
# --- check revision of the repo
git -C protonfixes rev-parse HEAD
# 681411ba8ceb5d2d790e674eb7a5b98951d426e6

# --- fast-import into new repo
git init newrepo
git -C newrepo fast-import < exported.txt
# --- checking revision of the new repo
git -C newrepo rev-parse HEAD
# 9888762d7857d9721f0c354e7fc187a199754a4b

Hashes don't match. The roundtrip fails.


Let's see if --reference-excluded-parents helps.

# --- export below produces the same export stream as above
git -C protonfixes fast-export --reference-excluded-parents --all >
exported_parents.txt


Because fast-import/fast-export don't work, you propose to keep the old
repo around until it is clear which commits I am going to modify. Then
make a new fast-export starting from the first commit I am going to
modify with --reference-excluded-parents flag. Is that correct so far?

Then given this partial export and old repo, how to init the new repo
that fast-import can apply its tail there?

What if I have multiple commits that I modify, but I don't know which
of their parents was first? And when I touch commits from different
branches, how to recreate their parent history intact in one repo?

-- 
anatoly t.

Re: Round-tripping fast-export/import changes commit hashes

From: Elijah Newren <hidden>
Date: 2023-01-13 07:27:24

On Sun, Dec 11, 2022 at 10:30 AM anatoly techtonik [off-list ref] wrote:
On Tue, Aug 10, 2021 at 8:58 PM Elijah Newren [off-list ref] wrote:
quoted
On Tue, Aug 10, 2021 at 8:51 AM anatoly techtonik [off-list ref] wrote:
quoted
On Mon, Aug 9, 2021 at 9:15 PM Elijah Newren [off-list ref] wrote:
quoted
[2] https://lore.kernel.org/git/CABPp-BH4dcsW52immJpTjgY5LjaVfKrY9MaUOnKT3byi2tBPpg@mail.gmail.com/

Signed commits is just one issue, and you'll have to add special code
to handle a bunch of other special cases if you go down this route.
I'd rephrase the problem.  You want to know when _your tool_ (e.g.
reposurgeon since you refer to it multiple times; I'm guessing you're
contributing to it?) has not modified a commit or any of its
ancestors, and when it hasn't, then _your tool_ should remove that
commit from the fast-export stream and replace any references to it by
the original commit's object id.  I outlined how to do this in [2],
referenced above, making use of the --show-original-ids flag to
fast-export.  If you do that, then for any commits which you haven't
modified (including not modifying any of its ancestors), then you'll
keep the same commits as-is with no stripping of gpg-signatures or
canonicalization of objects, so that you'll have the exact same commit
IDs.  Further, you can do this today, without any changes to git
fast-export or git fast-import.
Took me a while to process the reply. Let's recap.

I want to make a roundtrip export/import of
https://github.com/simons-public/protonfixes which should get exactly
the same repository.
As I've stated a few times in the thread, this request of yours is
simply impossible for general repositories ([1] contains the best
summary of the reasons).  For the specific repository in question, the
only relevant roadblocker is the presence of a signed commit which
happens to be a root commit.  That opens the door to some workarounds
that could be used with this specific repository.

[1] https://lore.kernel.org/git/CABPp-BGDB6jj+Et44D6D22KXprB89dNpyS_AAu3E8vOCtVaW1A@mail.gmail.com/

I provided two workarounds you could try to use for your specific case
at [2] and [3], one of which you ask about below.

[2] https://lore.kernel.org/git/CABPp-BE=9wzF6_VypoR-uEPHsLWdV7zyE13FOgLK0h8NOcMz3g@mail.gmail.com/
[3]  https://lore.kernel.org/git/CABPp-BH4dcsW52immJpTjgY5LjaVfKrY9MaUOnKT3byi2tBPpg@mail.gmail.com/
# --- fast-export to exported.txt
git clone https://github.com/simons-public/protonfixes
git -C protonfixes fast-export --all > exported.txt
# --- check revision of the repo
git -C protonfixes rev-parse HEAD
# 681411ba8ceb5d2d790e674eb7a5b98951d426e6

# --- fast-import into new repo
git init newrepo
git -C newrepo fast-import < exported.txt
# --- checking revision of the new repo
git -C newrepo rev-parse HEAD
# 9888762d7857d9721f0c354e7fc187a199754a4b

Hashes don't match. The roundtrip fails.
As expected, given that one of your commits is signed.
Let's see if --reference-excluded-parents helps.

# --- export below produces the same export stream as above
git -C protonfixes fast-export --reference-excluded-parents --all >
exported_parents.txt
--reference-excluded-parents only has effect if there are excluded
parents.  You didn't exclude any parents, so obviously adding this
flag isn't going to change anything.  You should instead first
clone/fetch the part of history up to the commits you want to keep
intact (e.g. the signed commits), and then run a command like
   git -C protonfixes fast-export --reference-excluded-parents
^${BASECOMMIT1} ^${BASECOMMIT2} ^${BASECOMMITN} --all
exported_only_newer_history.txt | git -C newrepo fast-import
Note that the examples I gave you (e.g. [2] above) all used some
excluded references (e.g. "^master~5").
Because fast-import/fast-export don't work
You have not yet identified a bug in either, so I disagree with this comment.
, you propose to keep the old
repo around until it is clear which commits I am going to modify.

This statement framing looks really weird to me.  You have posed your
problem in the form of doing some kind of export/import operation,
which is fine.  However, in order to do an export operation, you
obviously need the repository in order to export it.  So why are you
calling out that you keep the repo around until you run the
fast-export command?

Anyway, that aside...

I was just saying that
  (1) signed commits exist as a method to ensure to other users that
the commits have not been modified
  (2) fast-export and fast-import exist to allow you to modify history
in some fashion (and are separate steps so people can edit the stream
between running the two commands)
  (3) the above two imply that if you still want users to be able to
verify the signed commits, that signed commits should NOT be sent
through fast-export and fast-import
  (4) therefore, if you want the signed commits kept as-is, you should
simply fetch the history up to and including those, and only send the
remainder of the history through fast-export/fast-import.

But I will add here one additional thing:

If you're weaving repositories together, that likely changes the
parent(s) of some of the commits.  Once you change the parent(s) of a
commit, that alone changes the commit and invalidates any signature it
has.  In your case you seem to only have a root commit that is signed,
and if you keep that signed commit as a root commit, you can avoid
this problem.  But, in general, if signed commits are involved in the
weaving such that they gain new parents, then what you want to do is
simply impossible; you will not be able to keep the signatures in such
a case (and the commit ids will change as well).
Then
make a new fast-export starting from the first commit I am going to
modify with --reference-excluded-parents flag. Is that correct so far?
You have the basic idea, but you are making things excessively complex
with one detail here -- it does not need to start with the first
commit you are going to modify; it can start earlier.  You can simply
export all commits after the one(s) you know you don't want to change.
For example, if the history looks like this:

A---B---C---D---E---F

and commits A and B are the only signed commits (which you want to
preserve) and commit D is the first one you are going to modify, you
could still run fast-export on "^A ^B F" (i.e. C, D, E, and F in this
case) -- that will also include C, but C isn't signed and round-trips
without problems, so it doesn't hurt to include it.
Then given this partial export and old repo, how to init the new repo
that fast-import can apply its tail there?
Flag the signed commit(s) with a branch or branches of some sort, then
fetch just those branches into the new repo.
What if I have multiple commits that I modify, but I don't know which
of their parents was first?
I wouldn't bother trying to figure out which one(s) is/are first.  (I
mean, you could do some revision walking to figure that out, in which
case you'd have to fetch more than just the history of the signed
commits you want to keep but everything prior to whatever first
commit(s) you want to modify.)

Instead, I'd just do the easier thing I noted above -- use the signed
commits as exclusion markers.
And when I touch commits from different
branches, how to recreate their parent history intact in one repo?
Place temporary branches pointing directly to each of the signed
commits you want to keep intact (which also implies you are keeping
all the history behind those commits intact as well), then run:

git -C newrepo fetch PATH_OR_URL_OF_OLD_REPO ${TEMPBRANCH1}
${TEMPBRANCH2} ${TEMPBRANCHN}

Then use the earlier suggestion of

git -C protonfixes fast-export --reference-excluded-parents
^${TEMPBRANCH1} ^${TEMPBRANCH2} ^${TEMPBRANCHN} --all
exported_only_newer_history.txt | git -C newrepo fast-import
to get the remainder of the history exported/imported.



I will also add that since you are interested in attempting to
round-trip through fast-export/fast-import and still end up with the
same hashes (ignoring a few fundamental shortcomings mentioned earlier
in this thread that won't always permit this to work), you can at
least get closer by adding "--reencode=no" to fast-export (so that it
doesn't alter commit messages) and setting core.ignorecase=false for
at least the fast-import invocation (so that fast-import doesn't make
files which differ in case only clob each other while importing).
But, again, that only addresses like two issues out of half a dozen.
Again, see the link at [1] earlier in this email.
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help