Re: Round-tripping fast-export/import changes commit hashes

On Mon, Aug 9, 2021 at 9:15 PM Elijah Newren [off-list ref] wrote:

The author wasn't interested in implementing that
suggestion (and it's a low priority for me that I may never get around
to).  The series also wasn't pushed through and eventually was
dropped.

What it takes to validate the commit signature? Isn't it the same as
validating commit tag? Is it possible to merge at least the `--fast-export`
part? The effect of roundtrip would be the same, but at least external
tools would be able to detect signed commits and warn users.

[1] https://lore.kernel.org/git/20210430232537.1131641-1-lukeshu@lukeshu.com/ (local)

Yes, and I mentioned several other reasons why a round-trip from
fast-export through fast-import cannot be relied upon to preserve
object hashes.

Yes, I understand that. What would be the recommended way to detect
which commits would change as a result of the round-trip? It will then
be possible to warn users in `reposurgeon` `lint` command.

(3) fast-export works by looking for the relevant bits it knows how to
export.  You'd have to redesign it to fully parse every bit of data in
each object it looks at, throw errors if it didn't recognize any, and
make sure it exports all the bits.  That might be difficult since it's
hard to know how to future proof it.  How do you guarantee you've
printed every field in a commit struct, when that struct might gain
new fields in the future?  (This is especially challenging since
fast-export/fast-import might not be considered core tools, or at
least don't get as much attention as the "truly core" parts of git;
see https://lore.kernel.org/git/xmqq36mxdnpz.fsf@gitster-ct.c.googlers.com/ (local))

Looks like the only way to make it forward compatible is to introduce
some kind of versioning and a validation schema like protobuf. Otherwise
writing an importer and exporter for each and every thing that may
encounter in a git stream may be unrealistic, yes.

quoted

P.S. I am resurrecting the old thread, because my problem with editing
the history of the repository with an external tool still can not be solved.

Sure it can, just use fast-export's --reference-excluded-parents
option and don't export commits you know you won't need to change.

How does `--reference-excluded-parents` help to read signed commits?

`reposurgeon` needs all commits to select those that are needed by
different criteria. It is hard to tell which commits are not important without
reading and processing them first.

Or, if for some reason you are really set on exporting everything and
then editing, then go ahead and create the full fast-export output,
including with all your edits, and then post-process it manually
before feeding to fast-import.  In particular, in the post-processing
step find the commits that were problematic that you know won't be
modified, such as your signed commit.  Then go edit that fast-export
dump and (a) remove the dump of the no-longer-signed signed commit
(because you don't want it), and (b) replace any references to the
no-longer-signed-commit (e.g. "from :12") to instead use the hash of
the actual original signed commit (e.g. "from
d3d24b63446c7d06586eaa51764ff0c619113f09").  If you do that, then git
fast-import will just build the new commits on the existing signed
commit instead of on some new commit that is missing the signature.
Technically, you can even skip step (a), as all it will do is produce
an extra commit in your repository that isn't used and thus will be
garbage collected later.

The problem is to detect problematic signed commits, because as I
understand `fast-export` doesn't give any signs if commits were signed
before the export.
-- 
anatoly t.