From: Igor Djordjevic <hidden> Date: 2017-11-26 23:01:30
Hi to all,
Here`s my humble attempt on (once more) scratching a use case which
seems to be desired occasionally through the years, obviously not
enough to be actually implemented yet, but might be worth some more
attention... :)
For the reference, some past mentions of (more or less) similar ideas
are listed at the end of this e-mail[*1*], where I`m also Cc-ing
people previously involved, might be still sharing interest - as this
is my first "new thread" list message, I`m sorry if this is not an
encouraged practice, please let me know.
Approach discussed here could have a few more useful applications,
but one seems to be standing out the most - in case where multiple
topic branches are temporarily merged for integration testing, it
could be very useful to be able to post "hotfix" commits to merged
branches directly, _without actually switching to them_ (and thus
without touching working tree files), and still keeping everything
merged, in one go.
Example starting point is "master" branch with 3 topic branches (A,
B, C), to be (throwaway) merged for integration testing inside
temporary "test" branch:
(1) o---o---A (topicA)
/
/
/
---o---o---M (master, test, HEAD)
\ \
\ o---B (topicB)
\
o---o---C (topicC)
This is what we end up with once "master" and topic branches are
merged in merge commit M1 inside temporary "test" branch for further
integration testing:
(2) o---o---A (topicA)
/ \
/ M1 (test, HEAD)
/ /||
---o---o---M---/ || (master)
\ \ / |
\ o---B-/ | (topicB)
\ |
o---o---C--/ (topicC)
Upon discovery of a fix needed inside "topicA", hotfix changes X
should be committed to "topicA" branch and re-merged inside merge
commit M2 on temporary integration "test" branch (previous temporary
merge commit M1 is thrown away as uninteresting):
(3) o---o---A---X (topicA)
/ \
/ M2 (test, HEAD)
/ /||
---o---o---M-------/ || (master)
\ \ / |
\ o---B-----/ | (topicB)
\ /
o---o---C----/ (topicC)
Now, usually advised approach to get from (2) to (3), where one is
expected to switch to desired topic branch, commit the change, and
then re-merge everything again into integration testing branch, is
arguably a bit tiresome, but where it actually fails short is the
fact that branch switching back and forth (for each hotfix commit)
could possibly keep changing a lot of files otherwise untouched by
the hotfix changes we`re committing (but different between branches),
causing build systems to needlessly waste what could be significant
time compiling them again and again.
Example script proposes using something like this instead:
(4) git commit --onto-parent topicA
... in above-mentioned case (2) to commit X onto "topicA" branch
directly, re-merging all previously merged integration testing topic
branches at the same time, reaching state (3) without any
intermediate branch switching (and without touching working tree,
thus without needless recompilation).
Once integration tests pass, integration test branch will be thrown
away and new commits on each topic branch should still be properly
tested - we`re just deferring it not to do it in the middle of
integration testing, saving some (or a lot) needless rebuild cycles
at the same time (or in the first place, even).
Scripts in series:
[1/3]: setup.sh
[2/3]: git-merge-one-file--cached
[3/3]: git-commit--onto-parent.sh
Regards, Buga
[*1*] Some previous list mentions of similar functionality, in order
of appearance, latest on top (I kind of remember seeing more, but
can`t find them now, please feel free to add here, or notify more
people interested in the past):
- [PATCH/RFC] git-post: the opposite of git-cherry-pick (2017-10-05)
https://public-inbox.org/git/c6b52120-98bf-d685-6dc0-3c83e9e80d30@kdbg.org/
- "groups of files" in Git? (2017-07-11)
https://public-inbox.org/git/CAEcERAz3vYekvJ8SM1FfdAVsP3LMVqA1O3yoJVThvg-0fPtVCg@mail.gmail.com/
- Making a (quick) commit to another branch (2013-04-27)
https://public-inbox.org/git/517BDB6D.8040809@cedarsoft.com/
- Commit to other branch (2010-05-31)
https://public-inbox.org/git/4C03D9C1.1060404@cedarsoft.com/
- [RFC] git commit --branch (2006-05-29)
https://public-inbox.org/git/20060529202851.GE14325@admingilde.org/
- n-heads and patch dependency chains (2006-04-03)
https://public-inbox.org/git/4430D352.4010707@vilain.net/
- Multi-headed branches (hydra? :)) for basic patch calculus (2006-04-02)
https://public-inbox.org/git/1143950852.21233.23.camel@localhost.localdomain/
From: Igor Djordjevic <hidden> Date: 2017-11-26 23:01:40
On 26/11/2017 23:35, Igor Djordjevic wrote:
This is what we end up with once "master" and topic branches are
merged in merge commit M1 inside temporary "test" branch for further
integration testing:
(2) o---o---A (topicA)
/ \
/ M1 (test, HEAD)
/ /||
---o---o---M---/ || (master)
\ \ / |
\ o---B-/ | (topicB)
\ |
o---o---C--/ (topicC)
To begin with, you can use provided "setup.sh"[*1*] script, putting
you straight into position shown on graph (2) above, with addition of
tag "A" and remote branch "origin/topicA" so you could try using
these as "--onto-parent" values, too.
As seen in there, change "X" is already made and staged, so you can
now just run something like:
git-commit--onto-parent.sh --onto-parent topicA
... to see the logic in action.
Instead of "topicA", you may try providing tag "A", remote branch
"origin/topicA" or even plain commit hash. It`s interesting to add
"--amend" option into the mix, too, and see what happens. Also, you
can try using "topicB" and see the commit fail (as it doesn`t merge
cleanly).
All this while "test.txt" file doesn`t get modified on disk (nor
would any other file) - being desired behaviour, as we didn`t
actually change anything inside the working tree, but just amended
history of how we got here, so recompilation isn`t needlessly
triggered :)
p.s. Note these two lines near the end:
sed -i '4iX1' test.txt # works with simple patch apply
sed -i '17iX2' test.txt # needs three-way file merge
You can play with it, commenting out one or the other and observing
how it influences "git commit --onto-parent" in regards of the parent
provided.
Regards, Buga
[*1*] "setup.sh", can clean previous setup run as well, but commented
out here for safety, not to unexpectedly delete something for unwary
user.
--- 8< ---
#!/bin/sh
#rm -rf ./.git
#rm -f ./test.txt
git init
touch ./test.txt
git add -- test.txt
for i in {1..10}
do
echo $i >>test.txt
git commit -am "$i"
done
echo M >>test.txt
git commit -am "M"
git checkout -b topicA HEAD~2
for i in 1 2
do
sed -i "${i}iA${i}" test.txt
git commit -am "A$i"
done
sed -i '3iA' test.txt
git commit -am "A"
git tag A
# simulate remote branch
mkdir -p ./.git/refs/remotes/origin &&
echo $(git rev-parse HEAD^0) >$_/topicA
git checkout -b topicB master^
sed -i '4iB1' test.txt
git commit -am "B1"
sed -i '5iB' test.txt
git commit -am "B"
git checkout -b topicC master~2
for i in 1 2
do
j=`expr "$i" + 5`
sed -i "${j}iC${i}" test.txt
git commit -am "C$i"
done
sed -i "8iC" test.txt
git commit -am "C"
git checkout -b test master
git merge --no-edit topicA topicB topicC
sed -i '4iX1' test.txt # works with simple patch apply
sed -i '17iX2' test.txt # needs three-way file merge
git add -- test.txt
echo
git log --all --decorate --oneline --graph
echo
git diff --cached
From: Igor Djordjevic <hidden> Date: 2017-11-26 23:01:42
Finally, "git-commit--onto-parent.sh"[*1*] shows an initial script
version for you to examine, test out and hopefully comment on :)
Especially interesting part might be index-only three-way file merge,
through usage of "git-merge-one-file--cached" script. Of course, this
still only works for some trivial resolutions, where in case of more
complex ones, involving unresolved conflicts, we back-out and fail.
Still, it should be more powerful than `git-apply`.
Consider this proof of concept and work in progress, an idea where
I`d like feedback on everything you come up with or find interesting,
even parameter name possibly used instead of "--onto-parent" (and its
short version), or approach in general.
For example, it might make sense to separate commit creation (on
current HEAD`s parent) and its actual re-merging into integration
test branch, where "--remerge" (or something) parameter would be used
on top of "--onto-parent" to trigger both, if/when desired.
Another direction to think in might be introducing more general
"--onto" parameter, too (or instead), without "parent" restriction,
allowing to record a commit on top of any arbitrary commit (other
than HEAD). This could even be defaulted to "git commit <commit-ish>"
(no option needed), where current "git commit" behaviour would then
just be a special case of omitted <commit-ish> defaulting to HEAD,
aligning well with other Git commands sharing the same behaviour.
Alas, rewind to present...
Please do note that I`m still relatively new to Git, and pretty new
to both Linux and scripting in general (on Windows as well), and the
whole concept of open-source software contributing, even, so please
bare with me (or at least don`t get upset too much, lol), and do feel
free to share your thoughts and remarks, even the trivial or harsh
ones -- I`m grateful to learn and expand my knowledge, hopefully
producing something useful in return :) Heck, might be I`m totally
off-track here as well.
p.s. For some context - nowadays I mostly work in Delphi, and
occasionally in C#, though through last 20 years I`ve been involved
with C, Pascal, Basic, but also PHP, JavaScript, and whatnot - even
good old assembly from time to time, when needed :)
Regards, Buga
[*1*] "git-commit--onto-parent.sh", probably too heavily commented in
the first place, but as I`m new to everything here I kind of feel the
plain words might unfortunately describe my intention a bit better
than my code, for now at least.
--- 8< ---
#!/bin/sh
#
# Copyright (c) 2017 Igor Djordjevic
i=$#
while test $i != 0
do
#
# Parameter parsing might be uninteresting here, as the whole
# script is currently just a wrapper around `git commit`, for a
# functionality that conceptually belongs there directly.
#
case "$1" in
--onto-parent=*)
onto_parent="${1#*=}"
shift && i=$(expr $i - 1)
;;
--onto-parent)
shift && i=$(expr $i - 1)
onto_parent="$1"
shift && i=$(expr $i - 1)
;;
-a|--a|--al|--all)
all=t
#
# For now, `git commit` "--all" option is special-cased in
# terms of being stripped out of the original command line
# (to be passed to `git commit`) and processed manually, as
# once commit is to be made, due to states of index and
# working tree, "--all" is most probably NOT what the user
# wants nor expects ;)
#
shift && i=$(expr $i - 1)
;;
*)
# parameters to pass down to `git commit`
set -- "$@" "$1"
shift && i=$(expr $i - 1)
;;
esac
done
main () {
#
# Store current HEAD (ref or commit) and verify that
# --onto-parent is valid and amongst its parents.
#
head="$(git symbolic-ref --short --quiet HEAD)" ||
head="$(git rev-parse HEAD^0)" &&
verify_onto_parent "$head" "$onto_parent" || exit 1
#
# As both HEAD and "--onto-parent" could be refs, where underlying
# commits could change, store original commits for later parents
# processing, getting updated parent list for new/updated
# merge commit.
#
head_commit="$(git rev-parse "$head"^0)" &&
onto_parent_commit="$(git rev-parse "$onto_parent"^0)" || exit 1
#
# Custom processing of stripped "--all" parameter - if we were to
# just pass it to `git commit`, "--all" would most probably yield
# an unexpected result in the eyes of the user, as it would include
# _all changes from all the other merge commit parents as well_,
# not just the changes we may actually wanted to "push down"
# (commit) onto specified parent (what would `git diff` show),
# due to state of index and working tree at the time of commit.
#
if test -n "$all"
then
git add --update
fi
#
# Abort if no cached changes, nothing to be committed.
#
git diff --cached --quiet
if test $? -eq 0
then
printf >&2 '%s\n' "error: no changes added to commit"
exit 1
fi
#
# Backup current index to be restored (and committed) in the end.
#
merge_index="$(git write-tree)" || exit 1
#
# Reset index to destination parent (without touching working
# tree), and try applying cached changes.
#
# In case changes do not apply cleanly onto desired parent, abort.
#
apply_changes "$head_commit" "$onto_parent_commit" "$merge_index" "$onto_parent" || exit 1
#
# Move HEAD to specified parent (without touching working tree)
# to prepare to record a commit there, and make the commit,
# passing parameters through to `git commit`.
#
# In case of error, or when `git commit` didn`t actually make
# the commit, like when --dry-run parameter is provided (for
# example), abort, as there is nothing to do - no new commit, no
# need to produce the "updated" merge commit, either.
#
# Note that we don`t abort right away, as restoring original
# index and HEAD position is needed all the same, so we
# potentially abort only once that is done, a bit further below.
#
# [ This is something that could be thought of a bit more,
# might be forbidding passing through of some `git commit`
# parameters in the first place, like --dry-run...? ]
#
move_head "$onto_parent" &&
git commit "$@"
if test $? -ne 0 || {
new_parent_commit="$(git rev-parse HEAD^0)"
test "$new_parent_commit" = "$onto_parent_commit"
}
then
no_commit=t
fi
#
# Remove entry from HEAD reflog, not to pollute it with
# uninteresting in-between steps we take, leaking implementation
# details to end user.
#
# We do left it inside corresponding branch reflog where commit
# is made (if $onto_parent was a branch), though, as that`s where
# it still matters.
#
git reflog delete HEAD@{0}
#
# Restore original index state and move HEAD to original position,
# (still not touching working tree), aborting if previously
# signalled.
#
git read-tree --reset "$merge_index" &&
move_head "$head" &&
test -z "$no_commit" || exit 1
#
# Drop original HEAD merge commit to have it replaced by
# upcoming "updated" merge commit.
#
# This step is needed for eventually getting an expected merge
# message out of "git fmt-merge-msg", as it seems HEAD dependent
# as well, beside being input format picky already...?
#
#git update-ref --create-reflog -m "reset: moving to HEAD^" HEAD HEAD^ || exit 1
reflog_ref="$(git symbolic-ref --short --quiet HEAD)"
git update-ref HEAD HEAD^ || exit 1
#
# Remove both HEAD and underlying reference reflog entries this
# time, as here we really want to mask previous step completely,
# being taken just to satisfy "git fmt-merge-msg" expectations.
#
if test -n "$reflog_ref"
then
git reflog delete "$reflog_ref"@{0}
fi
git reflog delete HEAD@{0}
#
# Prepare "updated" merge commit message and parent list.
#
if test -n "$(git rev-parse --verify --quiet $head_commit^2^{commit})"
then
merge_parents="$(get_merge_parents "$head_commit" "$onto_parent" "$onto_parent_commit" "$new_parent_commit")" &&
merge_message="$(get_merge_message "$merge_parents")" || exit 1
else
#
# As we`re actually selling the option as "--onto-parent" and
# not "--onto-MERGE-parent", we might as well properly support
# a special case where HEAD commit is not a merge.
#
# Existing HEAD commit will come after commit to be made,
# basically being kind of rebased onto new commit (but still
# not touching working tree), where we can then also reuse
# original HEAD commit authorship, and message, too (instead
# of building a merge one).
#
merge_parents="$new_parent_commit" &&
merge_message="$(git show -s --format=%B "$head_commit")" &&
reuse_authorship $head_commit || exit 1
fi
merge_parent_commits="$(get_merge_parent_commits "$merge_parents")" &&
#
# Do the actual commit, updating HEAD accordingly.
#
merge_commit="$(printf '%s\n' "$merge_message" |
git commit-tree "$merge_index" $merge_parent_commits)" &&
git update-ref --create-reflog -m "$merge_message" HEAD "$merge_commit" || exit 1
}
verify_onto_parent () {
#
# $1 starting point head (ref or commit)
# $2 parent of $1 to commit onto (ref or commit)
#
local head="$1"
local onto_parent="$2"
if test -z "$onto_parent"
then
printf >&2 '%s\n' "error: no parent provided"
printf >&2 '%s\n' "(use \"--onto-parent <commit-ish>\""
return 1;
fi
if test -z "$(git rev-parse --verify --quiet "$onto_parent"^{commit})"
then
printf >&2 '%s\n' "error: '$onto_parent' not valid commit object"
return 1
fi
local onto_parent_commit="$(git rev-parse "$onto_parent"^0)"
for parent_commit in $(git rev-parse $head^@)
do
if test "$parent_commit" = "$onto_parent_commit"
then
return 0
fi
done
printf >&2 '%s\n' "error: '$onto_parent' not parent of '$head'"
return 1
}
apply_changes () {
#
# $1 original/starting point HEAD commit
# $2 parent commit of $1 to apply changes to
# $3 index with changes on top of $1 to apply/merge onto $2
# $4 original parameter value of $2 (ref or commit), used for
# prettier message only
#
local head_commit="$1"
local onto_parent_commit="$2"
local merge_index="$3"
local onto_parent="$4"
git read-tree --reset $onto_parent_commit &&
#
# Attempt simple patching first - take differences between
# $head_commit and $merge_index and try applying to current index
# (previously reset to $onto_parent_commit).
#
git diff-tree --binary --patch --find-renames --find-copies $head_commit $merge_index |
git apply --cached 2>/dev/null &&
return 0
printf '%s\n' "Unable to apply cleanly onto '$onto_parent', trying simple merge"
#
# A bit more aggressive approach - try merging with resolving
# trivial conflicts on tree level only (involving file as a whole,
# no conflicts inside file itself).
#
# Note that we take $head_commit as merge-base, producing such
# three-way merge result that basically all changes between
# $onto_parent_commit and $head_commit are reversed, as they`re
# also included inside $merge_index, where only differences
# between $head_commit and $merge_index are applied (in a
# three-way merge manner) to $onto_parent_commit, being exactly
# what we want here.
#
git read-tree -i -m --aggressive $head_commit $onto_parent_commit $merge_index || exit 1
git write-tree >/dev/null 2>&1 &&
return 0
printf '%s\n' "Simple merge did not work, trying automatic merge"
#
# Final attempt - try merging with resolving trivial conflicts on
# file level, too (conflicts inside file itself).
#
# Notice usage of "git-merge-one-file--cached" script here, being
# a slightly tweaked version of original "git-merge-one-file",
# not touching working tree but stuffing trivial three-way
# file merge resolution back into index directly.
#
# If still left with conflicts that need to be resolved manually,
# abort... and go home, you`re drunk.
#
if ! git merge-index -o git-merge-one-file--cached -a
then
# abort, cleanup
git read-tree --reset $merge_index
exit 1
fi
}
move_head () {
#
# $1 destination ref or commit
#
# Move HEAD to $1 without touching the working tree.
#
# Kind of "soft checkout", where original "git checkout" touches
# the working tree, and "git reset --soft" does not move HEAD,
# both undesired here.
#
local destination="$1"
local destination_commit="$(git rev-parse --verify --quiet $destination^0)" ||
{
printf >&2 '%s\n' "fatal: invalid reference: $destination"
return 1
}
#local reflog_message="$(get_checkout_reflog_message $destination)"
local destination_ref="$(git rev-parse --symbolic-full-name $destination)"
case "$destination_ref" in
refs/heads/*)
# can`t use "update-ref --no-deref" as it writes commit only,
# instead of ref, essentially detaching HEAD to that commit
#git symbolic-ref -m "$reflog_message" HEAD "$destination_ref"
git symbolic-ref HEAD "$destination_ref"
;;
refs/tags/*|\
refs/remotes/*|\
"")
# can`t use "symbolic-ref" as it refuses to write commit only,
# expecting a valid ref instead (value inside "refs/")
#git update-ref --create-reflog -m "$reflog_message" --no-deref HEAD "$destination_commit"
git update-ref --no-deref HEAD "$destination_commit"
# mask this step as end-user uninteresting implementation detail
git reflog delete HEAD@{0}
;;
*)
printf >&2 '%s\n' "fatal: invalid reference: $destination_ref"
return 1
;;
esac
return 0
}
get_merge_parents () {
#
# $1 original/starting point HEAD commit
# $2 parent of $1 to commit onto (ref or commit)
# $3 original commit of $2 (if $2 is ref, otherwise equals $2)
# $4 new commit (onto $2, to be new merge parent)
#
# Walk original merge commit parents to find the one we`re posting
# onto (or amending, even), and update/replace it accordingly with
# new commit, becoming a new parent of upcoming new/updated merge
# commit.
#
# Where possible, prefer taking ref over commit, making for a
# prettier merge commit message.
#
local head_commit="$1"
local onto_parent="$2"
local onto_parent_old_commit="$3"
local new_parent_commit="$4"
local merge_parents=
local onto_parent_new_commit="$(git rev-parse $onto_parent^0)"
for parent_commit in $(git rev-parse $head_commit^@)
do
local merge_parent=
if test "$parent_commit" = "$onto_parent_old_commit"
then
if test "$onto_parent_new_commit" = "$new_parent_commit"
then
# $onto_parent is a branch (updateable ref)
merge_parent="$onto_parent"
else
merge_parent="$new_parent_commit"
fi
else
parent_ref="$(git for-each-ref --points-at $parent_commit --count=1 --format="%(refname)")"
if test -n "$parent_ref"
then
merge_parent="$parent_ref"
else
merge_parent="$parent_commit"
fi
fi
# echo to flatten whitespace
merge_parents="$(echo $merge_parents $merge_parent)"
done
if test -n "$merge_parents"
then
printf '%s\n' "$merge_parents"
return 0
else
return 1
fi
}
get_merge_message () {
#
# $@ merge_parents
#
# Provide to-be merge commit message using
# existing `git fmt-merge-msg` machinery.
#
local merge_heads="$(get_merge_heads $@)" &&
local merge_message="$(printf "$merge_heads" | git fmt-merge-msg)"
if test -n "$merge_message"
then
printf '%s\n' "$merge_message"
return 0
else
return 1
fi
}
get_merge_heads () {
#
# $@ merge_parents
#
# Provide input for `git fmt-merge-msg` to get
# nicely formatted merge commit message.
#
# Final result loosely mimics FETCH_HEAD file layout.
#
local merge_heads=
local merge_head=
#
# Skip the first parent, as that is the original merge
# destination, where we`re only interested in parents to be
# merged into it.
#
shift
for merge_parent in $@
do
local merge_parent_ref="$(git rev-parse --symbolic-full-name $merge_parent)"
local merge_parent_commit="$(git rev-parse $merge_parent)"
case "$merge_parent_ref" in
refs/heads/*)
merge_head="$merge_parent_commit\t\tbranch '${merge_parent_ref#refs/heads/}' of ."
;;
refs/tags/*)
merge_head="$merge_parent_commit\t\ttag '${merge_parent_ref#refs/tags/}' of ."
;;
refs/remotes/*)
merge_head="$merge_parent_commit\t\tremote-tracking branch '${merge_parent_ref#refs/remotes/}' of ."
;;
*)
merge_head="$merge_parent_commit\t\t'$(git rev-parse --short $merge_parent_commit)' of ."
;;
esac
merge_heads="$merge_heads$merge_head\n"
done
if test -n "$merge_heads"
then
# '\n' already appended
printf '%s' "$merge_heads"
return 0
else
return 1
fi
}
reuse_authorship () {
#
# $1 commit to reuse authorship from
#
local commit="$1"
GIT_AUTHOR_NAME="$(git show -s --format=%an $commit)"
GIT_AUTHOR_EMAIL="$(git show -s --format=%ae $commit)"
GIT_AUTHOR_DATE="$(git show -s --format=%at $commit)"
export GIT_AUTHOR_NAME GIT_AUTHOR_EMAIL GIT_AUTHOR_DATE
}
get_merge_parent_commits () {
#
# $@ merge_parents (might contain ref)
#
# Provide to-be merge commit parent parameters
# in format suitable for `git commit-tree`.
#
local merge_parent_commits=
for merge_parent in $@
do
merge_parent_commits="$(printf '%s\n' "$merge_parent_commits -p $(git rev-parse $merge_parent^0)")"
done
if test -n "$merge_parent_commits"
then
printf '%s\n' "$merge_parent_commits"
return 0
else
return 1
fi
}
get_checkout_reflog_message () {
#
# $1 destination ref or commit
#
local destination="$1"
local source=
source="$(git symbolic-ref --short --quiet HEAD)" ||
source="$(git rev-parse HEAD^0)"
printf '%s' "checkout: moving from $source to $destination"
}
main "$@"
From: Igor Djordjevic <hidden> Date: 2017-11-26 23:01:43
Original "git-merge-one-file" script is slightly tweaked here into
"git-merge-one-file--cached"[*1*] to allow (still trivial) _index-only_
three-way file merge, not touching files inside working tree.
Proof of concept, not thoroughly tested, original content left in,
commented out. Feel free to comment/polish.
To make it available, I guess it would be best placed beside existing
"git-merge-one-file" script...?
Regards, Buga
[*1*] "git-merge-one-file--cached"
--- 8< ---
#!/bin/sh
#
# Copyright (c) Linus Torvalds, 2005
#
# ---
# Original "git-merge-one-file" script slightly tweaked into
# "git-merge-one-file--cached" to allow (still trivial) index-only
# three-way file merge, not touching files inside working tree.
#
# Proof of concept, not thoroughly tested, original content left in,
# commented out.
# ---
#
# This is the git per-file merge script, called with
#
# $1 - original file SHA1 (or empty)
# $2 - file in branch1 SHA1 (or empty)
# $3 - file in branch2 SHA1 (or empty)
# $4 - pathname in repository
# $5 - original file mode (or empty)
# $6 - file in branch1 mode (or empty)
# $7 - file in branch2 mode (or empty)
#
# Handle some trivial cases.. The _really_ trivial cases have
# been handled already by git read-tree, but that one doesn't
# do any merges that might change the tree layout.
USAGE='<orig blob> <our blob> <their blob> <path>'
USAGE="$USAGE <orig mode> <our mode> <their mode>"
LONG_USAGE="usage: git merge-one-file $USAGE
Blob ids and modes should be empty for missing files."
SUBDIRECTORY_OK=Yes
. git-sh-setup
cd_to_toplevel
require_work_tree
if test $# != 7
then
echo "$LONG_USAGE"
exit 1
fi
case "${1:-.}${2:-.}${3:-.}" in
#
# Deleted in both or deleted in one and unchanged in the other
#
"$1.." | "$1.$1" | "$1$1.")
if { test -z "$6" && test "$5" != "$7"; } ||
{ test -z "$7" && test "$5" != "$6"; }
then
echo "ERROR: File $4 deleted on one branch but had its" >&2
echo "ERROR: permissions changed on the other." >&2
exit 1
fi
if test -n "$2"
then
echo "Removing $4"
# else
# read-tree checked that index matches HEAD already,
# so we know we do not have this path tracked.
# there may be an unrelated working tree file here,
# which we should just leave unmolested. Make sure
# we do not have it in the index, though.
# exec git update-index --remove -- "$4"
fi
# if test -f "$4"
# then
# rm -f -- "$4" &&
# rmdir -p "$(expr "z$4" : 'z\(.*\)/')" 2>/dev/null || :
# fi &&
exec git update-index --remove --cacheinfo "$6","$2","$4"
;;
#
# Added in one.
#
".$2.")
# the other side did not add and we added so there is nothing
# to be done, except making the path merged.
exec git update-index --add --cacheinfo "$6","$2","$4"
;;
"..$3")
echo "Adding $4"
if test -f "$4"
then
echo "ERROR: untracked $4 is overwritten by the merge." >&2
exit 1
fi
git update-index --add --cacheinfo "$7","$3","$4" # &&
# exec git checkout-index -u -f -- "$4"
;;
#
# Added in both, identically (check for same permissions).
#
".$3$2")
if test "$6" != "$7"
then
echo "ERROR: File $4 added identically in both branches," >&2
echo "ERROR: but permissions conflict $6->$7." >&2
exit 1
fi
echo "Adding $4"
git update-index --add --cacheinfo "$6","$2","$4" # &&
# exec git checkout-index -u -f -- "$4"
;;
#
# Modified in both, but differently.
#
"$1$2$3" | ".$2$3")
case ",$6,$7," in
*,120000,*)
echo "ERROR: $4: Not merging symbolic link changes." >&2
exit 1
;;
*,160000,*)
echo "ERROR: $4: Not merging conflicting submodule changes." >&2
exit 1
;;
esac
src1=$(git unpack-file $2)
src2=$(git unpack-file $3)
case "$1" in
'')
echo "Added $4 in both, but differently."
orig=$(git unpack-file e69de29bb2d1d6434b8b29ae775ad8c2e48c5391)
;;
*)
echo "Auto-merging $4"
orig=$(git unpack-file $1)
;;
esac
git merge-file "$src1" "$orig" "$src2"
ret=$?
msg=
if test $ret != 0 || test -z "$1"
then
msg='content conflict'
ret=1
fi
# Create the working tree file, using "our tree" version from the
# index, and then store the result of the merge.
# git checkout-index -f --stage=2 -- "$4" && cat "$src1" >"$4" || exit 1
# rm -f -- "$orig" "$src1" "$src2"
if test "$6" != "$7"
then
if test -n "$msg"
then
msg="$msg, "
fi
msg="${msg}permissions conflict: $5->$6,$7"
ret=1
fi
if test $ret = 0
then
merge_blob=$(git hash-object -w --path="$4" -- "$src1")
fi
rm -f -- "$orig" "$src1" "$src2"
if test $ret != 0
then
echo "ERROR: $msg in $4" >&2
exit 1
fi
# exec git update-index -- "$4"
exec git update-index --cacheinfo "$6","$merge_blob","$4"
;;
*)
echo "ERROR: $4: Not handling case $1 -> $2 -> $3" >&2
;;
esac
exit 1
From: Johannes Sixt <hidden> Date: 2017-11-27 21:54:12
Am 26.11.2017 um 23:35 schrieb Igor Djordjevic:
Approach discussed here could have a few more useful applications,
but one seems to be standing out the most - in case where multiple
topic branches are temporarily merged for integration testing, it
could be very useful to be able to post "hotfix" commits to merged
branches directly, _without actually switching to them_ (and thus
without touching working tree files), and still keeping everything
merged, in one go.
Example starting point is "master" branch with 3 topic branches (A,
B, C), to be (throwaway) merged for integration testing inside
temporary "test" branch:
(1) o---o---A (topicA)
/
/
/
---o---o---M (master, test, HEAD)
\ \
\ o---B (topicB)
\
o---o---C (topicC)
This is what we end up with once "master" and topic branches are
merged in merge commit M1 inside temporary "test" branch for further
integration testing:
(2) o---o---A (topicA)
/ \
/ M1 (test, HEAD)
/ /||
---o---o---M---/ || (master)
\ \ / |
\ o---B-/ | (topicB)
\ |
o---o---C--/ (topicC)
Upon discovery of a fix needed inside "topicA", hotfix changes X
should be committed to "topicA" branch and re-merged inside merge
commit M2 on temporary integration "test" branch (previous temporary
merge commit M1 is thrown away as uninteresting):
(3) o---o---A---X (topicA)
/ \
/ M2 (test, HEAD)
/ /||
---o---o---M-------/ || (master)
\ \ / |
\ o---B-----/ | (topicB)
\ /
o---o---C----/ (topicC)
I my opinion, putting the focus on integration merge commits and the
desire to automate the re-merge step brings in a LOT of complexity in
the implementation for a very specific use-case that does not
necessarily help other cases.
For example, in my daily work, I have encountered situations where,
while working on one topic, I made a hot-fix for a different topic.
There is no demand for a merge step in this scenario.
In your scenario above, it would certainly not be too bad if you forgo
the automatic merge and have the user issue a merge command manually.
The resulting history could look like this:
(3) o---o---A---X (topicA)
/ \ \
/ M1--M2 (test, HEAD)
/ /||
---o---o---M---' || (master)
\ \ / |
\ o-----B / (topicB)
\ /
o---o---C (topicC)
I.e., commit --onto-parent A produced commit X, but M2 was then a
regular manual merge. (Of course, I am assuming that the merge commits
are dispensible, and only the resulting tree is of interest.)
Moreover, you seem to assume that an integration branch is an octopus
merge, that can be re-created easily. I would say that this a very, very
exceptional situation.
----
At this point, I spent five minutes thinking of how I would use commit
--onto-parent if I did not have git-post.
While on the integration branch, I typically make separate commits for
each fix, mostly because the bugs are discovered and fixed not
simultaneously, but over time. So, I have a small number of commits that
I distribute later using my git-post script. But that does not have to
be so. I think I could work with a git commit --onto-parent feature as
long as it does not attempt to make a merge commit for me. (I would hate
that.)
Sometimes, however I have two bug fixes in the worktree, ready to be
committed. Then the ability to pass pathspec to git commit is useful.
Does your implementation support this use case (partially staged
worktree changes)?
Thanks,
-- Hannes
From: Igor Djordjevic <hidden> Date: 2017-11-28 01:16:02
Hi Johannes,
On 27/11/2017 22:54, Johannes Sixt wrote:
I my opinion, putting the focus on integration merge commits and the
desire to automate the re-merge step brings in a LOT of complexity in
the implementation for a very specific use-case that does not
necessarily help other cases.
It might seem more complex than it is, until you examine the guts to
see how it really works :)
Basic concept is pretty simple, as I actually don`t automate
anything, at least not in terms of what would manual user steps look
like - for example, there`s no real re-merge step in terms of
actually redoing the merge, I just reuse what was already there in a
very clean way, I would think (supported by my current, humble
knowledge, still).
The only merge that could possibly ever happen is upon committing
desired subset of changes onto parent, and that shouldn`t be too
complex by definition, otherwise that commit doesn`t really belong
there in the first place, if it can`t be meaningfully applied where
we want it (for now, at least).
That said, the whole operation of "posting on parent and re-merging
everything", the way it looks like from the outside, could end just
with a simple diff-apply-commit-commit internally, no merges at all.
Only if simple `git apply` fails, we try some trivial merging - and
all that inside separate (parent) index only, not touching original
HEAD index nor working tree, staying pristine for the whole process,
untouched.
Once done, you should be in the very same situation you started from,
nothing changed, just having your history tweaked a bit to tell a
different story on how you got there (now including a commit you
posted on your HEAD`s parent).
Otherwise, I agree that explained use case might be a bit specific,
but that is only because I recognized that one to be the most
interesting to initially present (not to say one of more complex
cases) - to me, at least, but it is certainly not the only one.
Don`t let "usual/preferred/recommended" Git workflow distract you too
much - one of the reasons I made this is because it also allows _kind
of_ "vanilla Git" patch queue, where you can quickly work on top of
the merge head, pushing commits onto parents below, being tips of
your "queues", putting you up to speed without a need to ever switch
a branch (hypothetically), until satisfied with what you have, where
you can slow down and polish each branch separately, as usual.
Like working on multiple branches at the same time, in the manner
similar to what `git add --patch` allows in regards to working on
multiple commits at the same time. This just takes it on yet another
level... hopefully :)
For example, in my daily work, I have encountered situations where,
while working on one topic, I made a hot-fix for a different topic.
There is no demand for a merge step in this scenario.
In your scenario above, it would certainly not be too bad if you
forgo the automatic merge and have the user issue a merge command
manually. The resulting history could look like this:
(3) o---o---A---X (topicA)
/ \ \
/ M1--M2 (test, HEAD)
/ /||
---o---o---M---' || (master)
\ \ / |
\ o-----B / (topicB)
\ /
o---o---C (topicC)
I.e., commit --onto-parent A produced commit X, but M2 was then a
regular manual merge. (Of course, I am assuming that the merge
commits are dispensible, and only the resulting tree is of
interest.)
I see - and what you`re asking for is what I already envisioned and
hoped to get some more feedback about, here`s excerpt from
[SCRIPT/RFC 3/3] git-commit--onto-parent.sh[1] (I guess you didn`t
have time to checked that one yet?):
For example, it might make sense to separate commit creation (on
current HEAD`s parent) and its actual re-merging into integration
test branch, where "--remerge" (or something) parameter would be used
on top of "--onto-parent" to trigger both, if/when desired.
Another direction to think in might be introducing more general
"--onto" parameter, too (or instead), without "parent" restriction,
allowing to record a commit on top of any arbitrary commit (other
than HEAD). This could even be defaulted to "git commit <commit-ish>"
(no option needed), where current "git commit" behaviour would then
just be a special case of omitted <commit-ish> defaulting to HEAD,
aligning well with other Git commands sharing the same behaviour.
So I definitely look forward decoupling these two ((1) commit to
parent and (2) remerge), with enough discussion flowing :)
Heck, even "to parent" is an artificial/imposed restriction now, in
reality you could commit on top of any other commit you want (without
switching branches)... but let`s take one step at a time.
Just note that omitting the remerge step is what actually makes the
logic more complex, as we now need to change the original situation,
too, both HEAD index and working tree, to remove changes which we
committed elsewhere (without "merging" back in).
But it is interesting case to look into ;)
Moreover, you seem to assume that an integration branch is an octopus
merge, that can be re-created easily. I would say that this a very,
very exceptional situation.
Actually, I make no assumptions - head of "integration branch"
doesn`t even have to be a merge commit. And as we are not really
"remerging" anything, there is no need to recreate anything ;) (what
I tried to explain at the beginning of this e-mail).
The sole point is that your current situation inside HEAD doesn`t
change at all, no matter how complex it is, we just rewrite history
on how we got there (to include new commit onto desired parent).
At this point, I spent five minutes thinking of how I would use
commit --onto-parent if I did not have git-post.
While on the integration branch, I typically make separate commits
for each fix, mostly because the bugs are discovered and fixed not
simultaneously, but over time. So, I have a small number of commits
that I distribute later using my git-post script. But that does not
have to be so. I think I could work with a git commit --onto-parent
feature as long as it does not attempt to make a merge commit for me.
(I would hate that.)
I would be interested in looking into this, thanks for your feedback.
But, thinking about it more, you do realize that, without "updated
merge", that would be removing committed changes from where you`re
currently working at (once committed where you want it), is that what
you expect?
Because it seems a bit strange, as you loose overview on how all
those commits work together, and what you did so far, even, so some
future merge might yield unexpected conflicts, which could have been
avoided.
From what I understand, your `git-post` makes a commit where you are
_and copies_ it over to destination, so you end up with same changes
in two places (two commits). More, you seem to keep working on top of
your previous commit(s), which seems logical, later "posting" it
where desired - so after the fact. It also means you do still have
all your "fix" commits in HEAD while you work on them (and until you
git-post and abandon them, discarding the integration branch).
But `git commit --onto-parent` proposed here does the commit on
destination only, where later "remerge" is crucial part of still
keeping that commit changes in the current HEAD working tree as well,
so you can still keep working on top of them.
Could it be that you`re looking at `git commit --onto-parent` from an
unexpected perspective (through your `git-post`, being conceptually
different)...?
Though your use case graph (3) above should mean you`re very well
aware of implications, I guess, so I don`t know... :)
Sometimes, however I have two bug fixes in the worktree, ready to be
committed. Then the ability to pass pathspec to git commit is useful.
Does your implementation support this use case (partially staged
worktree changes)?
From what the idea is, it should work with anything that already
works with plain `git commit`, as script is currently just a wrapper
around it, providing additional functionality to speed things up.
So yes, `--onto-parent` works with `--patch`, `--amend`, etc :) If
not, that`s a bug which I`d appreciate being reported.
Might be some `git commit` options don`t have sense in combination
with `--onto-parent` as well, so those could be restricted, once
recognized.
... aaand that said, I`ve just noticed that pathspec doesn`t actually
work, for no good reason. If you add path through `git add` first, it
will work, but not through `git commit --onto-parent -- <pathspec>`
directly, which I need to address. Currently, it complains about no
changes added to commit. Thanks for spotting! ;)
From: Johannes Sixt <hidden> Date: 2017-11-29 19:11:13
Am 28.11.2017 um 02:15 schrieb Igor Djordjevic:
On 27/11/2017 22:54, Johannes Sixt wrote:
quoted
I my opinion, putting the focus on integration merge commits and the
desire to automate the re-merge step brings in a LOT of complexity in
the implementation for a very specific use-case that does not
necessarily help other cases.
It might seem more complex than it is, until you examine the guts to
see how it really works :)
Basic concept is pretty simple, as I actually don`t automate
anything, at least not in terms of what would manual user steps look
like - for example, there`s no real re-merge step in terms of
actually redoing the merge, I just reuse what was already there in a
very clean way, I would think (supported by my current, humble
knowledge, still).
The only merge that could possibly ever happen is upon committing
desired subset of changes onto parent, and that shouldn`t be too
complex by definition, otherwise that commit doesn`t really belong
there in the first place, if it can`t be meaningfully applied where
we want it (for now, at least).
That said, the whole operation of "posting on parent and re-merging
everything", the way it looks like from the outside, could end just
with a simple diff-apply-commit-commit internally, no merges at all.
Only if simple `git apply` fails, we try some trivial merging - and
all that inside separate (parent) index only, not touching original
HEAD index nor working tree, staying pristine for the whole process,
untouched.
Once done, you should be in the very same situation you started from,
nothing changed, just having your history tweaked a bit to tell a
different story on how you got there (now including a commit you
posted on your HEAD`s parent).
Ok, then please explain, how this process should work in my workflow and
with the `commit --onto-parent` feature that you have in mind. I have an
integration branch (which is a throw-away type, so you can mangle it in
any way you want); it is a series of merges:
...A ...C <- topics A, C
\ \
---o---o---o---o <- integration
/ /
...B ...D <- topics B, D
Now I find a bug in topic B. Assume that the merges of C and D have
textual conflicts with the integration branch (but not with B) and/or
may be evil. What should I do?
With git-post, I make a fixup commit commit on the integration branch,
then `git post B && git merge B`:
...A ...C <- topics A, C
\ \
---o---o---o---o---f---F <- integration
/ / /
...B ...D / <- topic D
\ /
f'----------' <- topic B
The merge F does not introduce any changes on the integration branch, so
I do not need it, but it helps keep topic B off radar when I ask `git
branch --no-merged` later.
Don`t let "usual/preferred/recommended" Git workflow distract you too
much - one of the reasons I made this is because it also allows _kind
of_ "vanilla Git" patch queue, where you can quickly work on top of
the merge head, pushing commits onto parents below, being tips of
your "queues", putting you up to speed without a need to ever switch
a branch (hypothetically), until satisfied with what you have, where
you can slow down and polish each branch separately, as usual.
Like working on multiple branches at the same time, in the manner
similar to what `git add --patch` allows in regards to working on
multiple commits at the same time. This just takes it on yet another
level... hopefully :)
'kay, I'm not eagerly waiting for this particular next level (I prefer
to keep things plain and simple), but I would never say this were a
broken workflow. ;)
quoted
In your scenario above, it would certainly not be too bad if you
forgo the automatic merge and have the user issue a merge command
manually. The resulting history could look like this:
(3) o---o---A---X (topicA)
/ \ \
/ M1--M2 (test, HEAD)
/ /||
---o---o---M---' || (master)
\ \ / |
\ o-----B / (topicB)
\ /
o---o---C (topicC)
I.e., commit --onto-parent A produced commit X, but M2 was then a
regular manual merge. (Of course, I am assuming that the merge
commits are dispensible, and only the resulting tree is of
interest.)
I see - and what you`re asking for is what I already envisioned and
hoped to get some more feedback about, here`s excerpt from
[SCRIPT/RFC 3/3] git-commit--onto-parent.sh[1] (I guess you didn`t
have time to checked that one yet?):
I did have a brief look, but I stopped when I saw
# Remove entry from HEAD reflog, not to pollute it with
# uninteresting in-between steps we take, leaking implementation
# details to end user.
It's a clear sign for me that's something wrong. It is not just reflogs
that can become stale, but all operations that follow the `git commit`
can fail. How do you clean up such a mess?
For example, it might make sense to separate commit creation (on
current HEAD`s parent) and its actual re-merging into integration
test branch, where "--remerge" (or something) parameter would be used
on top of "--onto-parent" to trigger both, if/when desired.
Another direction to think in might be introducing more general
"--onto" parameter, too (or instead), without "parent" restriction,
allowing to record a commit on top of any arbitrary commit (other
than HEAD). This could even be defaulted to "git commit <commit-ish>"
(no option needed), where current "git commit" behaviour would then
just be a special case of omitted <commit-ish> defaulting to HEAD,
aligning well with other Git commands sharing the same behaviour.
So I definitely look forward decoupling these two ((1) commit to
parent and (2) remerge), with enough discussion flowing :)
Heck, even "to parent" is an artificial/imposed restriction now, in
reality you could commit on top of any other commit you want (without
switching branches)... but let`s take one step at a time.
Just note that omitting the remerge step is what actually makes the
logic more complex, as we now need to change the original situation,
too, both HEAD index and working tree, to remove changes which we
committed elsewhere (without "merging" back in).
The "complex situation" I had in mind was not so much about the user's
view, but how the implementation has to work when there are errors.
quoted
From what I understand, your `git-post` makes a commit where you are
_and copies_ it over to destination, so you end up with same changes
in two places (two commits). More, you seem to keep working on top of
your previous commit(s), which seems logical, later "posting" it
where desired - so after the fact. It also means you do still have
all your "fix" commits in HEAD while you work on them (and until you
git-post and abandon them, discarding the integration branch).
Just to clarify: git-post only copies existing commits, but does not
make the original itself.
But `git commit --onto-parent` proposed here does the commit on
destination only, where later "remerge" is crucial part of still
keeping that commit changes in the current HEAD working tree as well,
so you can still keep working on top of them.
Could it be that you`re looking at `git commit --onto-parent` from an
unexpected perspective (through your `git-post`, being conceptually
different)...?
That may be the case. I'm very interested in your answer to my challenge
above.
-- Hannes
From: Igor Djordjevic <hidden> Date: 2017-11-29 23:10:33
Hi Hannes,
On 29/11/2017 20:11, Johannes Sixt wrote:
Ok, then please explain, how this process should work in my workflow
and with the `commit --onto-parent` feature that you have in mind. I
have an integration branch (which is a throw-away type, so you can
mangle it in any way you want); it is a series of merges:
...A ...C <- topics A, C
\ \ E
---o---o---o---o I <- integration
/ /
...B ...D <- topics B, D
Now I find a bug in topic B. Assume that the merges of C and D have
textual conflicts with the integration branch (but not with B) and/or
may be evil. What should I do?
With git-post, I make a fixup commit commit on the integration
branch, then `git post B && git merge B`:
...A ...C <- topics A, C
\ \
---o---o---o---o---f---F <- integration
/ / /
...B ...D / <- topic D
\ /
f'----------' <- topic B
The merge F does not introduce any changes on the integration branch,
so I do not need it, but it helps keep topic B off radar when I ask
`git branch --no-merged` later.
But you`re not committing (posting) on your HEAD`s (direct) parent in
the first place (topic B), so `commit --onto-parent` isn`t right tool
for the job... yet :)
To make it easier to explain, I marked your integration branch
initial head with "I" in the quote above (commit merging-in branch
D), and marked commit merging-in branch C with "E".
HEAD being currently on commit "I", you can only use `--onto-parent`
option to commit onto "E" or "D", being parents of "I".
To work with `--onto-parent` and be able to commit on top of any of
the topic branches, you would need a situation like this instead:
(1) ...C <- topic C
|
...A | <- topic A
\|
...o I <- integration
/|
...B | <- topic B
|
...D <- topic D
With `commit --onto-parent` you would skip `git post B && git merge
B` steps, where "fixup commit" would be done with `--onto-parent B`,
So you end up in situation like this:
(2) ...C <- topic C
|
...A | <- topic A
\|
...o I' <- integration
/|
...B---f | <- topic B
|
...D <- topic D
State of index and working tree files in your F and my I' commit is
exactly the same (I' = I + f), where in my case (2) history looks
like "f" was part of topic B from the start, before integration
merge happened.
BUT, all this said, I understand that your starting position, where
not all topic branches are merged at the same time (possibly to keep
conflict resolution sane), is probably more often to be encountered
in real-life work... and as existing `--onto-parent` machinery should
be able to support it already, I`m looking forward to make it happen :)
Once there, starting from your initial position:
...A ...C <- topics A, C
\ \ E
---o---o---o---o I <- integration <- HEAD
/ /
...B ...D <- topics B, D
... and doing something like `git commit --onto B --merge` would yield:
(3) ...A ...C <- topics A, C
\ \ E
---o---o---o---o I' <- integration
/ /|
...B ...D | <- topic D
\ |
f-------' <- topic B
... where (I' = I + f) is still true. If that`s preferred in some
cases, it could even look like this instead:
(4) ...A ...C <- topics A, C
\ \ E I
---o---o---o---o---F <- integration
/ / /
...B ...D / <- topic D
\ /
f-------' <- topic B
... where F(4) = I'(3), so similar situation, just that we don`t
discard I but post F on top of it.
Good thing is all necessary logic should already be in place, I just
need to think a bit about the most sane user interface, and get back
to you. Thanks for invaluable input so far :)
Of course, do feel free to drop any ideas you come up with as well,
on how `git commit` user interface/options leading to (3) or (4)
should look like (they could both be supported).
quoted
Like working on multiple branches at the same time, in the manner
similar to what `git add --patch` allows in regards to working on
multiple commits at the same time. This just takes it on yet another
level... hopefully :)
'kay, I'm not eagerly waiting for this particular next level (I
prefer to keep things plain and simple), but I would never say this
were a broken workflow. ;)
Hehe, thanks, I guess :) Simplicity, but user-oriented, is the major
point here, where you can work on unrelated patch series at the same
time (patch queues?), without a need to switch branches, while you
still make "per series" history as you go, without a need to do it
after the fact.
It`s a matter of possibilities (where Git usually excels), so one can
choose for himself, depending on the situation at hand.
quoted
[SCRIPT/RFC 3/3] git-commit--onto-parent.sh[1] (I guess you didn`t
have time to checked that one yet?):
I did have a brief look, but I stopped when I saw
# Remove entry from HEAD reflog, not to pollute it with
# uninteresting in-between steps we take, leaking implementation
# details to end user.
It's a clear sign for me that's something wrong. It is not just
reflogs that can become stale, but all operations that follow the
`git commit` can fail. How do you clean up such a mess?
I would respectfully disagree about it being wrong per se, but that
said, "cleaning up" reflogs was a last minute design decision so we
don`t show uninteresting states - from user`s point of view, all that
happened is commit on top of --onto-parent, and merging that commit
into where he already was (instead of the commit he was on).
Implementation details on how we got there should be irrelevant, even
more if we failed to do so, cluttering reflog with unimportant
leftovers.
Commands are chained, and in case _anything_ goes wrong, we just bail
out and restore original index and HEAD position. As working tree is
not touched at all, no cleanup needed there, so everything is pretty
simple. Of course, valid tests still should/need to be made, to make
sure everything is covered, but there really isn`t much to cover in
the first place.
But this is open for discussion, of course, and code for ref-logging
all steps is still inside the script, just commented out. Heck,
there`s even get_checkout_reflog_message() function at the end of the
script, used to mimic real checkout reflog message, that`s currently
unused ;)
The "complex situation" I had in mind was not so much about the
user's view, but how the implementation has to work when there are
errors.
Simple - abort. Restore backed up index and HEAD state is all that`s
to it, and it looks like we never issued the command to begin with.
quoted
From what I understand, your `git-post` makes a commit where you
are _and copies_ it over to destination, so you end up with same
changes in two places (two commits). More, you seem to keep working
on top of your previous commit(s), which seems logical, later
"posting" it where desired - so after the fact. It also means you
do still have all your "fix" commits in HEAD while you work on them
(and until you git-post and abandon them, discarding the
integration branch).
Just to clarify: git-post only copies existing commits, but does not
make the original itself.
Yes, I understand that (hopefully communicated from the second part
of the quote), but I see my wording of that first sentence was a bit
unfortunate there (to say the least, lol), thanks for clarifying.
quoted
Could it be that you`re looking at `git commit --onto-parent` from
an unexpected perspective (through your `git-post`, being
conceptually different)...?
That may be the case. I'm very interested in your answer to my
challenge above.
From everything said above, it seems so - at the moment, but I`m
looking forward making it work for you, too, as it does make sense
"out in the wild", in real-life scenarios.
Thank you again, for inspiring discussion.
Regards, Buga
From: Chris Nerwert <hidden> Date: 2017-11-30 22:40:33
On Nov 27, 2017, at 00:35, Igor Djordjevic [off-list ref] wrote:
Approach discussed here could have a few more useful applications,
but one seems to be standing out the most - in case where multiple
topic branches are temporarily merged for integration testing, it
could be very useful to be able to post "hotfix" commits to merged
branches directly, _without actually switching to them_ (and thus
without touching working tree files), and still keeping everything
merged, in one go.
I'm actually doing the described workflow quite often with git rebase when working on a topic. Given the following structure:
---o (master)
\
o---A---B---C (topic)
When I want to make changes to commit A one option is to make them directly on topic, then do "git commit --fixup A", and then eventual interactive rebase onto master will clean them up:
---o (master)
\
o---A---B---C---f!A (topic)
However, sometimes this breaks when changes in B or C conflict somehow with A (which may happen quite a lot during development of a topic), so the rebase will not apply cleanly. So sometimes I make a temporary branch from A, commit the fixup there:
---o (master)
\
o---A---B---C (topic)
\
f!A (temp)
and then use "git rebase --onto temp A topic" to move the topic back on track:
---o (master)
\
o---A---f!A (temp)
\
B'---C' (topic)
after which the final cleanup rebase is much easier to do.
Obviously, all the branch switching and rebasing does take its tall on file modifications.
From: Johannes Sixt <hidden> Date: 2017-12-01 17:23:58
Am 30.11.2017 um 00:10 schrieb Igor Djordjevic:
On 29/11/2017 20:11, Johannes Sixt wrote:
quoted
With git-post, I make a fixup commit commit on the integration
branch, then `git post B && git merge B`:
...A ...C <- topics A, C
\ \
---o---o---o---o---f---F <- integration
/ / /
...B ...D / <- topic D
\ /
f'----------' <- topic B
The merge F does not introduce any changes on the integration branch,
so I do not need it, but it helps keep topic B off radar when I ask
`git branch --no-merged` later.
But you`re not committing (posting) on your HEAD`s (direct) parent in
the first place (topic B), so `commit --onto-parent` isn`t right tool
for the job... yet :)
Ohh..kay.
To work with `--onto-parent` and be able to commit on top of any of
the topic branches, you would need a situation like this instead:
(1) ...C <- topic C
|
...A | <- topic A
\|
...o I <- integration
/|
...B | <- topic B
|
...D <- topic D
This is a very, VERY exotic workflow, I would say. How would you
construct commit I when three or more topics have conflicts?
merge-octopus does not support this use-case.
With `commit --onto-parent` you would skip `git post B && git merge
B` steps, where "fixup commit" would be done with `--onto-parent B`,
So you end up in situation like this:
(2) ...C <- topic C
|
...A | <- topic A
\|
...o I' <- integration
/|
...B---f | <- topic B
|
...D <- topic D
State of index and working tree files in your F and my I' commit is
exactly the same (I' = I + f), where in my case (2) history looks
like "f" was part of topic B from the start, before integration
merge happened.
BUT, all this said, I understand that your starting position, where
not all topic branches are merged at the same time (possibly to keep
conflict resolution sane), is probably more often to be encountered
in real-life work... and as existing `--onto-parent` machinery should
be able to support it already, I`m looking forward to make it happen :)
Once there, starting from your initial position:
quoted
...A ...C <- topics A, C
\ \ E
---o---o---o---o I <- integration <- HEAD
/ /
...B ...D <- topics B, D
... and doing something like `git commit --onto B --merge` would yield:
(3) ...A ...C <- topics A, C
\ \ E
---o---o---o---o I' <- integration
/ /|
...B ...D | <- topic D
\ |
f-------' <- topic B
... where (I' = I + f) is still true.
I am not used to this picture. I would not think that it is totally
unacceptable, but it still has a hmm-factor.
If that`s preferred in some
cases, it could even look like this instead:
(4) ...A ...C <- topics A, C
\ \ E I
---o---o---o---o---F <- integration
/ / /
...B ...D / <- topic D
\ /
f-------' <- topic B
... where F(4) = I'(3), so similar situation, just that we don`t
discard I but post F on top of it.
This is very acceptable.
Nevertheless, IMO, it is not the task of git-commit to re-compute a
merge commit. It would be OK that it commits changes on top of a branch
that is not checked out. Perhaps it would even be OK to remove the
change from the current workspace (index and worktree), because it will
return in the form of a merge later, but producing that merge is
certainly not the task of git-commit.
-- Hannes
From: Igor Djordjevic <hidden> Date: 2017-12-03 23:01:40
Hi Chris,
On 30/11/2017 23:40, Chris Nerwert wrote:
I'm actually doing the described workflow quite often with git rebase
when working on a topic. Given the following structure:
---o (master)
\
o---A---B---C (topic)
When I want to make changes to commit A one option is to make them
directly on topic, then do "git commit --fixup A", and then eventual
interactive rebase onto master will clean them up:
---o (master)
\
o---A---B---C---f!A (topic)
However, sometimes this breaks when changes in B or C conflict
somehow with A (which may happen quite a lot during development of a
topic), so the rebase will not apply cleanly. So sometimes I make a
temporary branch from A, commit the fixup there:
---o (master)
\
o---A---B---C (topic)
\
f!A (temp)
and then use "git rebase --onto temp A topic" to move the topic back
on track:
---o (master)
\
o---A---f!A (temp)
\
B'---C' (topic)
after which the final cleanup rebase is much easier to do.
Obviously, all the branch switching and rebasing does take its tall
on file modifications.
From use case you described (and which I often experience myself), it
seems plain "git commit --onto A" would be of help here, committing
fixup onto A directly, without a need to switch to it (branch or
not), a case I`m discussing with Hannes in that other sub-thread[1] of
this e-mail, too.
But from there, your flow takes a different direction, using rebase,
while this whole thread started around some merge-like functionality.
I can imagine a user interface doing what you (and I) would like,
something like:
(1) git commit --onto A --rebase
..., where your changes would first be committed onto commit A, and
then commits from A (excluded) to HEAD (included) rebased onto this
new commit.
BUT, as far as it seems to me, rebase currently touches working tree
for each operation (am I wrong here?), so once the rebase sequence is
initiated, it would internally still need to checkout to your new
fixup commit (on top of A), and then proceed applying changes and
changing working tree with each commit being rebased, overall failing
to address your main concern - needless (untouched) file
modifications, even in case of no conflicts.
I find this scenario quite interesting as well, but I`m afraid it may
currently be out of scope of what I`m trying to accomplish with "git
commit --onto[-parent]", for the most part because it looks like it
would need "index only rebase" first (not touching working tree, that
is)...?
If we had that, it would/should be pretty easy to add it into the mix
with "git commit --onto" here, ending up with something as imagined
in command line (1) above :) I`ll make a note of it, thanks.
Any further help appreciated, of course :)
Regards, Buga
[1] https://public-inbox.org/git/0F30BEF8-A9F7-43B9-BC89-4B9CD7AF3E16@gmail.com/T/#me830a80d745df60ae8bd6a2e67eee4bd4dabf56c
From: Igor Djordjevic <hidden> Date: 2017-12-04 02:33:43
Hi Hannes,
On 01/12/2017 18:23, Johannes Sixt wrote:
quoted
To work with `--onto-parent` and be able to commit on top of any of
the topic branches, you would need a situation like this instead:
(1) ...C <- topic C
|
...A | <- topic A
\|
...o I <- integration
/|
...B | <- topic B
|
...D <- topic D
This is a very, VERY exotic workflow, I would say. How would you
construct commit I when three or more topics have conflicts?
merge-octopus does not support this use-case.
But I`m not interested in constructing commit I in the first place,
only help working with it once it`s already there (which shouldn`t be
too uncommon in case of unrelated, non-conflicting topics) - but I
already agreed my example could have been a bit too esoteric,
distracting attention from the important part :)
I`ll continue on this further below, where you commented on those
more common scenarios.
Here, let`s try to look at the whole situation from a "patch queue"
perspective instead, starting from something like this:
(5) ---O---I <- integration <- HEAD
... and then making progress like this - first, commit A1 onto O,
starting "topicA" branch at the same time, even, maybe using syntax
like `git commit --onto-parent O -b topicA`:
(6) ---O---J <- integration <- HEAD [ J = I + A1 ]
\ /
A1 <- topicA
..., then commit B1 onto O to start "topicB" branch:
(7) B1 <- topicB
/ \
---O---K <- integration <- HEAD [ K = J + B1 ]
\ /
A1 <- topicA
..., then add one more commit (patch) onto B1:
(8) B1---B2 <- topicB
/ \
---O--------L <- integration <- HEAD [ L = K + B2 ]
\ /
A1---/ <- topicA
..., and one more, B3:
(9) B1---B2---B3 <- topicB
/ \
---O-------------M <- integration <- HEAD [ M = L + B3 ]
\ /
A1--------/ <- topicA
We can also start a new topic branch (queue), commit C1:
(10) B1---B2---B3 <- topicB
/ \
---O-------------N <- integration <- HEAD [ N = M + C1 ]
|\ /|
| A1--------/ / <- topicA
\ /
C1---------/ <- topicC
And lets make one more "topicA" related commit A2:
(11) B1---B2---B3 <- topicB
/ \
---O-------------P <- integration <- HEAD [ P = N + A2 ]
|\ /|
| A1---A2---/ / <- topicA
\ /
C1---------/ <- topicC
Notice how HEAD never leaves "integration" branch, and underlying
commit is recreated each time? It`s like a live branch we`re working
on, but we`re not actually committing to.
No branch switching (and no working tree file changes caused by it),
and we`re working on multiple topics/branches simultaneously, being
able to instantly test their mutual interaction as we go, but also
creating their separate (and "clean") histories at the same time.
I guess this would make most sense with simple topics we _could_
practically work on at the same time without making our life too
complicated - what stands for "git add --patch", too, when working on
multiple commits at the same time.
Once satisfied, of course each topic branch would need to be tested
separately, and each commit, even - all the same as with "git add
--patch" commits.
And "git add --patch" can still be used here, too, to distribute
partial changes, currently existing together inside the working tree,
to different topic branches, at the time of making the commit itself.
Does this approach make more sense in regards to "git commit
--onto-parent" functionality I`m having in mind? Or I`m dreaming too
much here...? :)
quoted
Once there, starting from your initial position:
quoted
...A ...C <- topics A, C
\ \ E
---o---o---o---o I <- integration <- HEAD
/ /
...B ...D <- topics B, D
... and doing something like `git commit --onto B --merge` would
yield:
(3) ...A ...C <- topics A, C
\ \ E
---o---o---o---o I' <- integration
/ /|
...B ...D | <- topic D
\ |
f-------' <- topic B
... where (I' = I + f) is still true.
I am not used to this picture. I would not think that it is totally
unacceptable, but it still has a hmm-factor.
Main idea is not to pile up uninteresting merge commits inside
(throwaway) integration branch, needlessly complicating history, but
pretend as we just made the integration merge, where fixup commit f
was already existing in its topic branch prior the merge.
quoted
If that`s preferred in some
cases, it could even look like this instead:
(4) ...A ...C <- topics A, C
\ \ E I
---o---o---o---o---F <- integration
/ / /
...B ...D / <- topic D
\ /
f-------' <- topic B
... where F(4) = I'(3), so similar situation, just that we don`t
discard I but post F on top of it.
This is very acceptable.
And I can see logic behind this case, too (alongside that other one,
where they can both be supported, for different scenarios).
Nevertheless, IMO, it is not the task of git-commit to re-compute a
merge commit. It would be OK that it commits changes on top of a
branch that is not checked out. Perhaps it would even be OK to remove
the change from the current workspace (index and worktree), because
it will return in the form of a merge later, but producing that merge
is certainly not the task of git-commit.
Just to make sure we`re on the same page - you mean conceptually, or
practically? Or both...? :)
Because practically, we don`t do any merge operations/computations
(as can be seen inside the script), so we really don`t produce any
merges :)
Having a merge commit being updated as an outcome is just a
consequence of altering our history to pretend fixup commit already
existed elsewhere, being brought into our current state from there -
and that "current state" is updated accordingly to reflect the change
we made in the history that led to it.
Let`s look at a different example, no merges involved:
(12) ---A---B---C---D <- HEAD
Here, we can do `git commit --onto-parent C` to make the history look
like this:
(13) ---A---B---C---E---D' <- HEAD
..., where we practically inserted commit E between our HEAD and
commit C, causing D to be updated to D' accordingly (rebased,
practically).
Idea is the same with merges involved, updating HEAD commit to
reflect altered history.
In situation (12), we could also do `git commit --onto-parent C
--amend`, ending up with history like this instead:
(14) ---A---B---F---D' <- HEAD
..., where F is amended C, E being added to it, instead of being a
separate commit.
So there are quite some possibilities here.
But as said, I can understand a wish to have a commit elsewhere
without "merging" it in, too (changes removed from current workspace),
or "merging" it inside a separate commit (not updating existing
HEAD one).
And saying that, making the merge commit at the same time could then
be observed as no more than a shortcut, too, like being able to
create a new branch on checkout using "-b" option.
So even conceptually, it would make sense to wish for something like
this:
(15) git commit --onto B --merge
..., producing that graph (4) shown in the quote above, I would
think?
But even in that case, do note no merge is really happening, we just
alter history of our current state, where we already are.
Splitting changes out of HEAD to a commit directly preceding it, on a
different branch, and still keeping those changes in HEAD, naturally
causes our current HEAD state to have multiple parents, what is
perceived as a merge commit, even without a merge operation leading
to it.
Thus, `git commit` seems it should be up to the task, whichever
approach user opts for :)
Regards, Buga
From: Johannes Sixt <hidden> Date: 2017-12-06 18:34:59
I am sorry for not responding in detail. I think we've reached a mutual
understanding of our workflows.
Though, from the ideas you tossed around most recently, you seem to want
to make git-commit into a kitchen-sink for everything. I have my doubts
that this will be a welcome change. Just because new commits are created
does not mean that the feature must live in git-commit.
-- Hannes
From: Junio C Hamano <hidden> Date: 2017-12-06 18:41:09
On Wed, Dec 6, 2017 at 10:34 AM, Johannes Sixt [off-list ref] wrote:
I am sorry for not responding in detail. I think we've reached a mutual
understanding of our workflows.
Though, from the ideas you tossed around most recently, you seem to want to
make git-commit into a kitchen-sink for everything. I have my doubts that
this will be a welcome change. Just because new commits are created does not
mean that the feature must live in git-commit.
From: Igor Djordjevic <hidden> Date: 2017-12-08 00:15:49
On 06/12/2017 19:34, Johannes Sixt wrote:
I am sorry for not responding in detail. I think we've reached a
mutual understanding of our workflows.
No problem, thanks for your time so far.
There might be one more thing I should address, possibly left unclear
from my previous message, but I`ll leave that for a follow-up e-mail,
not being that important at the moment for the topic itself.
On 06/12/2017 19:40, Junio C Hamano wrote:
quoted
Though, from the ideas you tossed around most recently, you seem to
want to make git-commit into a kitchen-sink for everything. I have
my doubts that this will be a welcome change. Just because new
commits are created does not mean that the feature must live in
git-commit.
Nicely put.
Yeah, I understand that might have felt cluttering, besides also
being out of scope of the original topic idea. Thanks for the reality
check (to both).
To get back on track, and regarding what`s already been said, would
having something like this(1) feel useful?
(1) git commit --onto <commit>
So in previously mentioned situation:
(2) ...A ...C <- topics A, C
\ \
---o---o---o---o I <- integration <- HEAD
/ /
...B ...D <- topics B, D
... it would allow committing changes F inside HEAD on top of B
directly, no checkout / branch switching needed, getting to:
(3) ...A ...C <- topics A, C
\ \
---o---o---o---o I <- integration <- HEAD
/ /
...B ...D <- topic D
\
F <- topic B
So the most conservative approach, where changes F are removed from
HEAD index and working tree, leaving it up to the user to decide if
he will then merge them back in (or do something else).
I stress the major selling point here still being avoiding branch
switching back and forth in order to commit a fixup on a different
branch, which could otherwise trigger needless rebuilds, being
significant in large projects.
And thanks to that `git-merge-one-file--cached`[1] script, we are
also able to resolve some more of trivial conflicts when applying F
onto B, using three-way file merge when needed, but still not
touching working tree (contrary to original `git-merge-one-file`).
Regards, Buga
[1] https://public-inbox.org/git/CAPc5daWupO6DMOMFGn=XjUCG-JMYc4eyo8+TmAsdWcAOHXzwWg@mail.gmail.com/T/#mcb3953542dc265516e3ab1bff006ff1b5b85126a