What's cooking in git.git (Jun 2021, #07; Wed, 30)

Here are the topics that have been cooking in my tree.  Commits
prefixed with '-' are only in 'seen' while commits prefixed with '+'
are in 'next'.  The ones marked with '.' do not appear in any of the
integration branches, but I am still holding onto them.  Generally,
being in 'next' is a sign that a topic is stable enough to be used
and are candidate to be in a future release, while being in 'seen'
means nothing more than that the maintainer has found it interesting
for some reason (like "it may have hard-to-resolve conflicts with
another topic already in flight" or "this may turn out to be
useful")---do not read too much into a topic being in (or not in)
'seen'.

Some new topics have been merged to 'seen' and are listed in this
report but there may well be topics that I haven't got around to
pick them up.  Also, among those that are in 'seen', I haven't
summarized many of them in this rerpot (let alone decided what to do
with them).  Help and patience is greatly appreciated.

Copies of the source code to Git live in many repositories, and the
following is a list of the ones I push into or their mirrors.  Some
repositories have only a subset of branches.

With maint, master, next, seen, todo:

	git://git.kernel.org/pub/scm/git/git.git/
	git://repo.or.cz/alt-git.git/
	https://kernel.googlesource.com/pub/scm/git/git/
	https://github.com/git/git/
	https://gitlab.com/git-vcs/git/

With all the integration branches and topics broken out:

	https://github.com/gitster/git/

Even though the preformatted documentation in HTML and man format
are not sources, they are published in these repositories for
convenience (replace "htmldocs" with "manpages" for the manual
pages):

	git://git.kernel.org/pub/scm/git/git-htmldocs.git/

1206 more lines

Junio C Hamano wrote:

Here are the topics that have been cooking in my tree.

--------------------------------------------------
[New Topics]

This is a topic:

https://lore.kernel.org/git/20210623004815.1807-1-felipe.contreras@gmail.com/

So is this:

https://lore.kernel.org/git/20210615183157.104999-1-felipe.contreras@gmail.com/

And this:

https://lore.kernel.org/git/20210628031642.699156-1-felipe.contreras@gmail.com/

And this:

https://lore.kernel.org/git/20210607185043.893205-1-felipe.contreras@gmail.com/

-- 
Felipe Contreras

On Wed, Jun 30 2021, Junio C Hamano wrote:

* jh/builtin-fsmonitor (2021-05-24) 30 commits
 - t/perf: avoid copying builtin fsmonitor files into test repo
 - t7527: test status with untracked-cache and fsmonitor--daemon
 - p7519: add fsmonitor--daemon
 - t7527: create test for fsmonitor--daemon
 - fsmonitor: force update index after large responses
 - fsmonitor: enhance existing comments
 - fsmonitor--daemon: use a cookie file to sync with file system
 - fsmonitor--daemon: periodically truncate list of modified files
 - fsmonitor--daemon: implement handle_client callback
 - fsmonitor-fs-listen-macos: implement FSEvent listener on MacOS
 - fsmonitor-fs-listen-macos: add macos header files for FSEvent
 - fsmonitor-fs-listen-win32: implement FSMonitor backend on Windows
 - fsmonitor--daemon: create token-based changed path cache
 - fsmonitor--daemon: define token-ids
 - fsmonitor--daemon: add pathname classification
 - fsmonitor--daemon: implement daemon command options
 - fsmonitor-fs-listen-macos: stub in backend for MacOS
 - fsmonitor-fs-listen-win32: stub in backend for Windows
 - t/helper/fsmonitor-client: create IPC client to talk to FSMonitor Daemon
 - fsmonitor--daemon: implement client command options
 - fsmonitor--daemon: add a built-in fsmonitor daemon
 - fsmonitor: introduce `core.useBuiltinFSMonitor` to call the daemon via IPC
 - config: FSMonitor is repository-specific
 - help: include fsmonitor--daemon feature flag in version info
 - fsmonitor-ipc: create client routines for git-fsmonitor--daemon
 - fsmonitor--daemon: update fsmonitor documentation
 - fsmonitor--daemon: man page
 - simple-ipc: preparations for supporting binary messages.
 - Merge branch 'jk/perf-in-worktrees' into HEAD
 - Merge branch 'jh/simple-ipc' into jh/rfc-builtin-fsmonitor

 An attempt to write and ship with a watchman equivalent tailored
 for our use.

 What's the status of this one?

I think Johannes's reply to the last WC applies[1]:

    I am not Jeff, but I know that he is busy getting back to it, and
    plans on submitting a third iteration.

FWIW I'm still curious about some details on the performance concerns
that seem to have prompted this built-in fsmonitor endeavor, as I asked
about (but didn't get a reply to) in [2].

Not as a "we shouldn't have this, let's keep using the hook", but just
curiosity about why we've seemingly gotten such different performance
numbers on the watchman hook v.s. a built-in approach.

I suspect (but don't know) that the reason is that the built-in is
perhaps integrating differently with git somehow, in a way that we could
retrofit the hook approach to also do (if anyone still cares about the
hook approach).

In any case I'm interested in *why* the new approach is faster, given
that I've done some testing (again, noted in [2]) that suggest that
bottleneck in the previous pipeline wasn't at all what Jeff H. thought
it was.

1. https://lore.kernel.org/git/nycvar.QRO.7.76.6.2106171135530.57@tvgsbejvaqbjf.bet/#t
2. https://lore.kernel.org/git/87h7lgfchm.fsf@evledraar.gmail.com/

On 7/1/21 9:42 AM, Ævar Arnfjörð Bjarmason wrote:

On Wed, Jun 30 2021, Junio C Hamano wrote:

quoted

* jh/builtin-fsmonitor (2021-05-24) 30 commits
  - t/perf: avoid copying builtin fsmonitor files into test repo
  - t7527: test status with untracked-cache and fsmonitor--daemon
  - p7519: add fsmonitor--daemon
  - t7527: create test for fsmonitor--daemon
  - fsmonitor: force update index after large responses
  - fsmonitor: enhance existing comments
  - fsmonitor--daemon: use a cookie file to sync with file system
  - fsmonitor--daemon: periodically truncate list of modified files
  - fsmonitor--daemon: implement handle_client callback
  - fsmonitor-fs-listen-macos: implement FSEvent listener on MacOS
  - fsmonitor-fs-listen-macos: add macos header files for FSEvent
  - fsmonitor-fs-listen-win32: implement FSMonitor backend on Windows
  - fsmonitor--daemon: create token-based changed path cache
  - fsmonitor--daemon: define token-ids
  - fsmonitor--daemon: add pathname classification
  - fsmonitor--daemon: implement daemon command options
  - fsmonitor-fs-listen-macos: stub in backend for MacOS
  - fsmonitor-fs-listen-win32: stub in backend for Windows
  - t/helper/fsmonitor-client: create IPC client to talk to FSMonitor Daemon
  - fsmonitor--daemon: implement client command options
  - fsmonitor--daemon: add a built-in fsmonitor daemon
  - fsmonitor: introduce `core.useBuiltinFSMonitor` to call the daemon via IPC
  - config: FSMonitor is repository-specific
  - help: include fsmonitor--daemon feature flag in version info
  - fsmonitor-ipc: create client routines for git-fsmonitor--daemon
  - fsmonitor--daemon: update fsmonitor documentation
  - fsmonitor--daemon: man page
  - simple-ipc: preparations for supporting binary messages.
  - Merge branch 'jk/perf-in-worktrees' into HEAD
  - Merge branch 'jh/simple-ipc' into jh/rfc-builtin-fsmonitor

  An attempt to write and ship with a watchman equivalent tailored
  for our use.

  What's the status of this one?

26 more quoted lines

A quick reply here of Junio's question. Yes, I'm working on a V3
to submit (any day now -- $DAYJOB notwithstanding (read: meetings)).

I'll push this up and then try to answer the perf questions.

Thanks for your patience.
Jeff

Comments on my outstanding:

On Wed, Jun 30 2021, Junio C Hamano wrote:

* ab/bundle-doc (2021-06-30) 3 commits
 - bundle doc: elaborate on rev<->ref restriction
 - bundle doc: split out open v.s. closed discussion from <rev-arg>
 - bundle doc: rewrite the "DESCRIPTION" section

 Doc update.

Will update per your https://lore.kernel.org/git/xmqqk0mbt5rj.fsf@gitster.g/

* ab/pack-stdin-packs-fix (2021-06-28) 2 commits
 - pack-objects: fix segfault in --stdin-packs option
 - pack-objects tests: cover blindspots in stdin handling

 Input validation of "git pack-objects --stdin-packs" has been
 corrected.

 cf. [off-list ref]

Will address & re-roll.

* ab/bundle-updates (2021-06-30) 3 commits
 - bundle: remove "ref_list" in favor of string-list.c API
 - bundle.c: use a temporary variable for OIDs and names
 - bundle cmd: stop leaking memory from parse_options_cmd_bundle()


* ab/fetch-negotiate-segv-fix (2021-06-30) 3 commits
 - fetch: fix segfault in --negotiate-only without --negotiation-tip=*
 - fetch: document the --negotiate-only option
 - send-pack.c: move "no refs in common" abort earlier

Has outstanding comments, will re-roll.

* ab/make-delete-on-error (2021-06-29) 1 commit
 - Makefile: add and use the ".DELETE_ON_ERROR" flag

I think this is stable at this point & has gotten a lot of
attention/review:
https://lore.kernel.org/git/patch-1.1-2557117855-20210629T084356Z-avarab@gmail.com/

* ab/make-tags-cleanup (2021-06-29) 5 commits
 - Makefile: normalize clobbering & xargs for tags targets
 - Makefile: don't use "FORCE" for tags targets
 - Makefile: fix "cscope" target to refer to cscope.out
 - Makefile: add QUIET_GEN to "cscope" target
 - Makefile: move ".PHONY: cscope" near its target

Will re-roll per https://lore.kernel.org/git/67c45b13-df8f-8065-377a-fbd2f80992ee@ramsayjones.plus.com/

* ab/describe-tests-fix (2021-05-11) 5 commits
 - describe tests: support -C in "check_describe"
 - describe tests: fix nested "test_expect_success" call
 - describe tests: don't rely on err.actual from "check_describe"
 - describe tests: refactor away from glob matching
 - describe tests: improve test for --work-tree & --dirty

 Various updates to tests around "git describe"

 Will merge to 'next'.

Thanks!

* ab/pickaxe-pcre2 (2021-05-11) 22 commits
 - xdiff-interface: replace discard_hunk_line() with a flag
 - xdiff users: use designated initializers for out_line
 - pickaxe -G: don't special-case create/delete
 - pickaxe -G: terminate early on matching lines
 - xdiff-interface: allow early return from xdiff_emit_line_fn
 - xdiff-interface: prepare for allowing early return
 - pickaxe -S: slightly optimize contains()
 - pickaxe: rename variables in has_changes() for brevity
 - pickaxe -S: support content with NULs under --pickaxe-regex
 - pickaxe: assert that we must have a needle under -G or -S
 - pickaxe: refactor function selection in diffcore-pickaxe()
 - perf: add performance test for pickaxe
 - pickaxe/style: consolidate declarations and assignments
 - diff.h: move pickaxe fields together again
 - pickaxe: die when --find-object and --pickaxe-all are combined
 - pickaxe: die when -G and --pickaxe-regex are combined
 - pickaxe tests: add missing test for --no-pickaxe-regex being an error
 - pickaxe tests: test for -G, -S and --find-object incompatibility
 - pickaxe tests: add test for "log -S" not being a regex
 - pickaxe tests: add test for diffgrep_consume() internals
 - pickaxe tests: refactor to use test_commit --append --printf
 - grep/pcre2 tests: reword comments referring to kwset

 Rewrite the backend for "diff -G/-S" to use pcre2 engine when
 available.

 Will merge to 'next'.

Thanks!

* ab/config-based-hooks-base (2021-06-29) 33 commits
 - hooks: fix a TOCTOU in "did we run a hook?" heuristic
 - receive-pack: convert receive hooks to hook.h
 - post-update: use hook.h library
 - receive-pack: convert 'update' hook to hook.h
 - hooks: allow callers to capture output
 - run-command: allow capturing of collated output
 - reference-transaction: use hook.h to run hooks
 - transport: convert pre-push hook to use config
 - hook: convert 'post-rewrite' hook in sequencer.c to hook.h
 - hook: provide stdin by string_list or callback
 - run-command: add stdin callback for parallelization
 - am: convert 'post-rewrite' hook to hook.h
 - hook: support passing stdin to hooks
 - run-command: allow stdin for run_processes_parallel
 - run-command: remove old run_hook_{le,ve}() hook API
 - receive-pack: convert push-to-checkout hook to hook.h
 - read-cache: convert post-index-change hook to use config
 - commit: use hook.h to execute hooks
 - git-p4: use 'git hook' to run hooks
 - send-email: use 'git hook run' for 'sendemail-validate'
 - git hook run: add an --ignore-missing flag
 - merge: use config-based hooks for post-merge hook
 - hooks: convert 'post-checkout' hook to hook library
 - am: convert applypatch hooks to use config
 - rebase: teach pre-rebase to use hook.h
 - gc: use hook library for pre-auto-gc hook

For this one I'd submitted these independently as

 - hook: add 'run' subcommand
 - hook-list.h: add a generated list of hooks, like config-list.h
 - hook.c: add a hook_exists() wrapper and use it in bugreport.c
 - hook.[ch]: move find_hook() to this new library

https://lore.kernel.org/git/cover-0.3-0000000000-20210629T183325Z-avarab@gmail.com/

(René had a comment about his SOB on a patch there, but I see you
addressed it while queuing it (changed to Helped-by))

And:

 - Makefile: remove an out-of-date comment
 - Makefile: stop hardcoding {command,config}-list.h
 - Makefile: mark "check" target as .PHONY

https://lore.kernel.org/git/cover-0.3-0000000000-20210629T190137Z-avarab@gmail.com/

But you re-combined them. If it makes things easier for you I'm all for
it, but it seems better to me keep them split if that mean that some
parts can advance faster, and thus make the rest easier to review.

 Restructuring of (a subset of) Emily's config-based-hooks series,
 to demonstrate that a series can be presented as a more logical and
 focused progression.

 Waiting for reviews.

I think it's mainly waiting on Emily giving it a detailed look, per
https://lore.kernel.org/git/YNEoMZ1QhZwmS0v2@google.com/

* ab/config-hooks-path-testfix (2021-06-16) 1 commit
 - pre-commit hook tests: don't leave "actual" nonexisting on failure

 Test fix.

 Will merge to 'next'.


* ab/doc-retire-alice-bob (2021-06-16) 6 commits
 - pack-protocol doc: use "www-data" in place of "alice"
 - doc: replace "alice" and "bob" with "jdoe" and "msmith"
 - fast-import doc: change "bob" in an example to "file.txt"
 - daemon doc + code comments: reword "alice" example
 - gitcvs-migration doc: replace "alice" and "bob" with "you" and "www-data"
 - gittutorial doc: replace "alice" and "bob" with "you" and "www-data"

Having re-read the discussion now I don't know if there's anything
outstanding to change about this series. It's gotten a lot of attention
so far, so it's more of a matter of if you're willing to take this sort
of documentation change or not.

* ab/serve-cleanup (2021-06-28) 8 commits
 - upload-pack.c: convert to new serve.c "startup" config cb
 - serve: add support for a "startup" git_config() callback
 - serve.c: add trace2 regions for advertise & command
 - serve.c: add call_{advertise,command}() indirection
 - serve: use designated initializers
 - transport: use designated initializers
 - transport: rename "fetch" in transport_vtable to "fetch_refs"
 - serve: mark has_capability() as static

 Code clean-up around "git serve".

Jeff, what do you think about this version & the upload-pack.c changes?

* ab/mktag-tests (2021-06-28) 6 commits
 - mktag tests: test fast-export
 - mktag tests: test for-each-ref
 - mktag tests: test update-ref and reachable fsck
 - mktag tests: test hash-object --literally and unreachable fsck
 - mktag tests: invert --no-strict test
 - mktag tests: parse out options in helper

 Fill test gaps.


* ab/show-branch-tests (2021-06-28) 4 commits
 - show-branch tests: add missing tests
 - show-branch: don't <COLOR></RESET> for space characters
 - show-branch tests: modernize test code
 - show-branch tests: rename the one "show-branch" test file

 Fill test gaps.

I think both of these are ready for next.

The mktag one had trivial improvement suggestions I've addressed in a
re-roll already. Felipe gave v2 of ab/show-branch-tests a review here:
https://lore.kernel.org/git/60cbbe9d89cbd_1299208fa@natae.notmuch/

I'm partial to just keeping it as-is, I think in 2/4 it's OK to fix
various test modernization issues at once & they don't need to be split
up into more commits, it's easy enough to follow along with it. What do
you think?

* ab/cmd-foo-should-return (2021-06-09) 1 commit
 - builtins + test helpers: use return instead of exit() in cmd_*

 Code clean-up.

 Will merge to 'next'.


* ab/progress-cleanup (2021-06-08) 1 commit
 - read-cache.c: don't guard calls to progress.c API

 Code clean-up.

 Will merge to 'next'.

Thanks!

* ab/test-tool-cache-cleanup (2021-06-08) 4 commits
 - read-cache perf: add a perf test for refresh_index()
 - test-tool: migrate read-cache-again to parse_options()
 - test-tool: migrate read-cache-perf to parse_options()
 - test-tool: split up test-tool read-cache

 Test code shuffling.

 Waiting for reviews.

Emily gave this a review, waiting on her reply to my reply at
https://lore.kernel.org/git/87pmw3dlkl.fsf@evledraar.gmail.com/

I.e. whether the outstanding is to merely clarify more in commit
messages that the goal is to split up a doing-to-many-things function,
not to optimize an existing perf test.

* ab/xdiff-bug-cleanup (2021-06-08) 1 commit
 - xdiff: use BUG(...), not xdl_bug(...)

 Code clean-up.

 Will merge to 'next'.

Thanks!

* ab/pack-objects-stdin (2021-06-28) 4 commits
 - pack-objects.c: make use of REV_INFO_STDIN_LINE_PROCESS
 - pack-objects.c: do stdin parsing via revision.c's API
 - revision.h: refactor "disable_stdin" and "read_from_stdin"
 - upload-pack: run is_repository_shallow() before setup_revisions()

 Code clean-up.

Emily & Jonathan Tan gave the version preceding this one a detailed
once-over, my re-roll addressed their concerns, but it would be good to
hear if they agree with that assessment.

* ab/update-submitting-patches (2021-06-08) 3 commits
 - SubmittingPatches: remove pine-specific hints from MUA hints
 - SubmittingPatches: replace discussion of Travis with GitHub Actions
 - SubmittingPatches: move discussion of Signed-off-by above "send"

 Reorganize and update the SubmitingPatches document.

 Expecting a reroll.
 cf. [off-list ref]
 cf. [off-list ref]

Will try to re-roll sooner than later...

* ab/send-email-optim (2021-05-28) 13 commits
 - perl: nano-optimize by replacing Cwd::cwd() with Cwd::getcwd()
 - send-email: move trivial config handling to Perl
 - perl: lazily load some common Git.pm setup code
 - send-email: lazily load modules for a big speedup
 - send-email: get rid of indirect object syntax
 - send-email: use function syntax instead of barewords
 - send-email: lazily shell out to "git var"
 - send-email: lazily load config for a big speedup
 - send-email: copy "config_regxp" into git-send-email.perl
 - send-email: refactor sendemail.smtpencryption config parsing
 - send-email: remove non-working support for "sendemail.smtpssl"
 - send-email tests: test for boolean variables without a value
 - send-email tests: support GIT_TEST_PERL_FATAL_WARNINGS=true

 "git send-email" optimization.

 Will merge to 'next'.

Thanks!

* ab/fsck-unexpected-type (2021-06-28) 21 commits
 - fsck: report invalid object type-path combinations
 - fsck: report invalid types recorded in objects
 - object-store.h: move read_loose_object() below 'struct object_info'
 - fsck: don't hard die on invalid object types
 - object-file.c: return -2 on "header too long" in unpack_loose_header()
 - object-file.c: return -1, not "status" from unpack_loose_header()
 - object-file.c: guard against future bugs in loose_object_info()
 - object-file.c: stop dying in parse_loose_header()
 - object-file.c: split up ternary in parse_loose_header()
 - object-file.c: simplify unpack_loose_short_header()
 - object-file.c: add missing braces to loose_object_info()
 - object-file.c: make parse_loose_header_extended() public
 - object-file.c: don't set "typep" when returning non-zero
 - cache.h: move object functions to object-store.h
 - cat-file tests: test for current --allow-unknown-type behavior
 - cat-file tests: add corrupt loose object test
 - rev-list tests: test for behavior with invalid object types
 - cat-file tests: test that --allow-unknown-type isn't on by default
 - cat-file tests: test for missing object with -t and -s
 - fsck tests: add test for fsck-ing an unknown type
 - fsck tests: refactor one test to use a sub-repo

 "git fsck" has been taught to report mismatch between expected and
 actual types of an object better.

There's one trivial comment on the v4, will re-roll as a v5:
https://lore.kernel.org/git/87mtrezya9.fsf@evledraar.gmail.com/

I think this should be ready to move to 'next', but it would be good to
get other eyes on the mid-series changes I made in v4:
https://lore.kernel.org/git/cover-00.21-00000000000-20210624T191754Z-avarab@gmail.com/

On Wed, Jun 30, 2021 at 03:55:33PM -0700, Junio C Hamano wrote:

* jk/test-avoid-globmatch-with-skip-patterns (2021-06-17) 1 commit
 - test-lib: avoid accidental globbing in match_pattern_list()

 We broke "GIT_SKIP_TESTS=t?000" to skip certain tests in recent
 update, which got fixed.

 Will merge to 'next'.

Not a big deal, but the commit message needs s/t?0000/t?000/ (the
numbers are four-digits, and the "?" is one of them). If it's not too
much trouble before it hits 'next'. :)

-Peff

On Thu, Jul 01, 2021 at 04:00:01PM +0200, Ævar Arnfjörð Bjarmason wrote:

quoted

* ab/serve-cleanup (2021-06-28) 8 commits
 - upload-pack.c: convert to new serve.c "startup" config cb
 - serve: add support for a "startup" git_config() callback
 - serve.c: add trace2 regions for advertise & command
 - serve.c: add call_{advertise,command}() indirection
 - serve: use designated initializers
 - transport: use designated initializers
 - transport: rename "fetch" in transport_vtable to "fetch_refs"
 - serve: mark has_capability() as static

 Code clean-up around "git serve".

Jeff, what do you think about this version & the upload-pack.c changes?

I left some comments on the config-callback patches. I think it's an OK
direction.

I didn't read the final patch carefully. And I do think it needs to be
gone over very carefully to make sure there are not any subtleties,
given the problems we've had with config-reading there in the past.  I'm
not enthused about doing so for what I consider to be a pretty
unexciting outcome. I.e., it feels a bit like high-risk churn for low
reward.

-Peff

On Wed, Jun 30, 2021 at 3:58 PM Junio C Hamano [off-list ref] wrote:

I've started reviewing v7 of ds/status-with-sparse-index and will also
look at ds/commit-and-checkout-with-sparse-index afterwards.

* en/ort-perf-batch-13 (2021-06-28) 5 commits
 - merge-ort: add prefetching for content merges
 - diffcore-rename: use a different prefetch for basename comparisons
 - diffcore-rename: allow different missing_object_cb functions
 - t6421: add tests checking for excessive object downloads during merge
 - promisor-remote: output trace2 statistics for number of objects fetched

 Performance tweaks of "git merge -sort" around lazy fetching of objects.

Most of the discussion on this series was actually about related
improvements and ideas, with both Stolee and I agreeing those belong
outside this particular series.

I also tried to ask Jonathan Tan to review since it was based upon his
similar work to do the original prefetching in diffcore-rename, but I
think he's been busy.

I'm not aware of further tweaks needed right now, but it's not at all
clear to me whether I addressed your and Stolee's feedback to each of
your satisfaction.  So this one could potentially be marked "Waiting
for reviews (to finalize)"

* en/zdiff3 (2021-06-15) 2 commits
 - update documentation for new zdiff3 conflictStyle
 - xdiff: implement a zealous diff3, or "zdiff3"

 "Zealous diff3" style of merge conflict presentation has been added.

 Expecting a reroll.
 cf. [off-list ref]

I've got some updates for this RFC series (in part thanks to Phillip's
awesome digging!), including some tests.  However, I've also got a
clear testcase failure, and another
I-know-I-saw-a-real-problem-in-real-testing-but-lost-the-details
issue.  So I may wait a while to resend.

* pw/diff-color-moved-fix (2021-06-15) 10 commits
 - diff --color-moved: intern strings
 - diff --color-moved-ws=allow-indentation-change: improve hash lookups
 - diff --color-moved: stop clearing potential moved blocks
 - diff --color-moved: shrink potential moved blocks as we go
 - diff --color-moved: unify moved block growth functions
 - diff --color-moved: call comparison function directly
 - diff --color-moved-ws=allow-indentation-change: simplify and optimize
 - diff: simplify allow-indentation-change delta calculation
 - diff --color-moved: avoid false short line matches and bad zebra coloring
 - diff --color-moved=zebra: fix alternate coloring

 Long-overdue correctness and performance update to "diff
 --color-moved" feature.

 Waiting for reviews.

I read through the first half the patches or so and didn't spot
anything wrong, though I'm very unfamiliar with this code area and
don't yet understand much of what's going on; it was more of a spot
check.  I'll try to get back to it again, but my review probably won't
be that deep.

* en/ort-perf-batch-12 (2021-06-09) 4 commits
 - merge-ort: miscellaneous touch-ups
 - Fix various issues found in comments
 - diffcore-rename: avoid unnecessary strdup'ing in break_idx
 - merge-ort: replace string_list_df_name_compare with faster alternative

 More fix-ups and optimization to "merge -sort".

 Will merge to 'next'.

Sounds good, thanks.  As far as I can tell, I addressed everyone's
feedback a month or more ago (including getting Stolee's Reviewed-by);
no changes since then.

* ab/send-email-optim (2021-05-28) 13 commits
 - perl: nano-optimize by replacing Cwd::cwd() with Cwd::getcwd()
 - send-email: move trivial config handling to Perl
 - perl: lazily load some common Git.pm setup code
 - send-email: lazily load modules for a big speedup
 - send-email: get rid of indirect object syntax
 - send-email: use function syntax instead of barewords
 - send-email: lazily shell out to "git var"
 - send-email: lazily load config for a big speedup
 - send-email: copy "config_regxp" into git-send-email.perl
 - send-email: refactor sendemail.smtpencryption config parsing
 - send-email: remove non-working support for "sendemail.smtpssl"
 - send-email tests: test for boolean variables without a value
 - send-email tests: support GIT_TEST_PERL_FATAL_WARNINGS=true

 "git send-email" optimization.

 Will merge to 'next'.

Wahoo!

On Fri, Jul 02 2021, Elijah Newren wrote:

On Wed, Jun 30, 2021 at 3:58 PM Junio C Hamano [off-list ref] wrote:
[...]

quoted

* ab/send-email-optim (2021-05-28) 13 commits
 - perl: nano-optimize by replacing Cwd::cwd() with Cwd::getcwd()
 - send-email: move trivial config handling to Perl
 - perl: lazily load some common Git.pm setup code
 - send-email: lazily load modules for a big speedup
 - send-email: get rid of indirect object syntax
 - send-email: use function syntax instead of barewords
 - send-email: lazily shell out to "git var"
 - send-email: lazily load config for a big speedup
 - send-email: copy "config_regxp" into git-send-email.perl
 - send-email: refactor sendemail.smtpencryption config parsing
 - send-email: remove non-working support for "sendemail.smtpssl"
 - send-email tests: test for boolean variables without a value
 - send-email tests: support GIT_TEST_PERL_FATAL_WARNINGS=true

 "git send-email" optimization.

 Will merge to 'next'.

Wahoo!

Do you find the faster t9001*.sh helps your overall test-run time?

On Mon, Jul 05, 2021 at 10:03:58AM +0200, Ævar Arnfjörð Bjarmason wrote:

quoted

quoted

* ab/send-email-optim (2021-05-28) 13 commits
 - perl: nano-optimize by replacing Cwd::cwd() with Cwd::getcwd()
 - send-email: move trivial config handling to Perl
 - perl: lazily load some common Git.pm setup code
 - send-email: lazily load modules for a big speedup
 - send-email: get rid of indirect object syntax
 - send-email: use function syntax instead of barewords
 - send-email: lazily shell out to "git var"
 - send-email: lazily load config for a big speedup
 - send-email: copy "config_regxp" into git-send-email.perl
 - send-email: refactor sendemail.smtpencryption config parsing
 - send-email: remove non-working support for "sendemail.smtpssl"
 - send-email tests: test for boolean variables without a value
 - send-email tests: support GIT_TEST_PERL_FATAL_WARNINGS=true

 "git send-email" optimization.

 Will merge to 'next'.

Wahoo!

Do you find the faster t9001*.sh helps your overall test-run time?

It does for me. Running "make test" on 'next' (with -O2, and prove
--state=slow,save so it realizes t9001 is slow) takes 66 wallclock
seconds total. The top culprits are:

  $ perl -MYAML -e ' 
      $_ = do { local $/; <> };
      # prove puts this non-YAML cruft at the end
      s/\.\.\.$//s;
      my $t = YAML::Load($_)->{tests};
      print "$_->[1] $_->[0]\n" for
        sort { $b->[1] <=> $a->[1] } 
        map { [$_, $t->{$_}->{elapsed}] } 
        keys(%$t);
    ' t/.prove  | head
  65.4168658256531 t9001-send-email.sh
  40.6761560440063 t0027-auto-crlf.sh
  26.0225791931152 t7610-mergetool.sh
  24.0760719776154 t7112-reset-submodule.sh
  23.5710120201111 t5572-pull-submodule.sh
  23.2557609081268 t9500-gitweb-standalone-no-errors.sh
  20.9747018814087 t1701-racy-split-index.sh
  20.7937531471252 t7400-submodule-basic.sh
  20.4510779380798 t5510-fetch.sh
  20.0246639251709 t3305-notes-fanout.sh

Merging in ab/send-email-optim takes 60s, and yields:

  39.4541878700256 t0027-auto-crlf.sh
  27.2213349342346 t7610-mergetool.sh
  26.2227098941803 t9001-send-email.sh
  25.0272631645203 t5572-pull-submodule.sh
  24.3502569198608 t7112-reset-submodule.sh
  24.0568630695343 t9500-gitweb-standalone-no-errors.sh
  22.3555839061737 t7400-submodule-basic.sh
  21.2626190185547 t3305-notes-fanout.sh
  21.1138088703156 t5510-fetch.sh
  21.0040950775146 t1701-racy-split-index.sh

So we shaved 39s of CPU (albeit clocked-down-due-to-parallelism

5 more lines

On 03/07/2021 06:04, Elijah Newren wrote:

On Wed, Jun 30, 2021 at 3:58 PM Junio C Hamano [off-list ref] wrote:

quoted

* pw/diff-color-moved-fix (2021-06-15) 10 commits
  - diff --color-moved: intern strings
  - diff --color-moved-ws=allow-indentation-change: improve hash lookups
  - diff --color-moved: stop clearing potential moved blocks
  - diff --color-moved: shrink potential moved blocks as we go
  - diff --color-moved: unify moved block growth functions
  - diff --color-moved: call comparison function directly
  - diff --color-moved-ws=allow-indentation-change: simplify and optimize
  - diff: simplify allow-indentation-change delta calculation
  - diff --color-moved: avoid false short line matches and bad zebra coloring
  - diff --color-moved=zebra: fix alternate coloring

  Long-overdue correctness and performance update to "diff
  --color-moved" feature.

  Waiting for reviews.

I read through the first half the patches or so and didn't spot
anything wrong, though I'm very unfamiliar with this code area and
don't yet understand much of what's going on; it was more of a spot
check.  I'll try to get back to it again, but my review probably won't
be that deep.

Thanks for taking a look, do let me know if I can do anything to make reviewing this series easier

Best Wishes

Phillip

On Tue, Jul 06 2021, Phillip Wood wrote:

On 03/07/2021 06:04, Elijah Newren wrote:

quoted

On Wed, Jun 30, 2021 at 3:58 PM Junio C Hamano [off-list ref] wrote:

quoted

* pw/diff-color-moved-fix (2021-06-15) 10 commits
  - diff --color-moved: intern strings
  - diff --color-moved-ws=allow-indentation-change: improve hash lookups
  - diff --color-moved: stop clearing potential moved blocks
  - diff --color-moved: shrink potential moved blocks as we go
  - diff --color-moved: unify moved block growth functions
  - diff --color-moved: call comparison function directly
  - diff --color-moved-ws=allow-indentation-change: simplify and optimize
  - diff: simplify allow-indentation-change delta calculation
  - diff --color-moved: avoid false short line matches and bad zebra coloring
  - diff --color-moved=zebra: fix alternate coloring

  Long-overdue correctness and performance update to "diff
  --color-moved" feature.

  Waiting for reviews.

I read through the first half the patches or so and didn't spot
anything wrong, though I'm very unfamiliar with this code area and
don't yet understand much of what's going on; it was more of a spot
check.  I'll try to get back to it again, but my review probably won't
be that deep.

Thanks for taking a look, do let me know if I can do anything to make
reviewing this series easier

FWIW I put it on my own mental list of "waiting for re-roll" per [1].

(Not that you need to take my suggestion there, just that you seemed to
agree, so I thought I'd take a look when that happened in a v2).

1. https://lore.kernel.org/git/094f5e5f-d447-8867-a9a7-be5c8827bba6@gmail.com/