Re: [PATCH v2 7/8] submodule-config: pass repo upon blob config read

On Fri, Aug 13, 2021 at 6:05 PM Jonathan Tan [off-list ref] wrote:

When reading the config of a submodule, if reading from a blob, read
using an explicitly specified repository instead of by adding the
submodule's ODB as an alternate and then reading an object from
the_repository.

Great!

At first, I thought this would also allow us to remove another
NEEDSWORK comment in grep_submodule(), together with a lock
protection:

/*
 * NEEDSWORK: repo_read_gitmodules() might call
 * add_to_alternates_memory() via config_from_gitmodules(). This
 * operation causes a race condition with concurrent object readings
 * performed by the worker threads. That's why we need obj_read_lock()
 * here. It should be removed once it's no longer necessary to add the
 * subrepo's odbs to the in-memory alternates list.
 */
obj_read_lock();
repo_read_gitmodules(subrepo, 0);

Back when I wrote this comment, my conclusion was that the alternates
mechanics were the only thread-unsafe object-reading operations in
repo_read_gitmodules()'s call chains. So once the add-to-alternates
mechanics were gone, we could also remove the lock.

But with further inspection now, I see that this is not really the
case. For example, we have a few global variables in packfile.c
collecting some statistics (pack_mmap_calls, pack_open_windows, etc.)
which are updated on obj readings from both the_repository *and*
submodules. So I no longer think its safe to remove the
obj_read_lock() protection here, as the NEEDSWORK comment suggests,
even if we are not using the alternates list anymore.

Do you want to remove this comment in your patchset? I can also send a
follow-up patch explaining this situation and removing the comment
(but not the locking), if you prefer.

Signed-off-by: Jonathan Tan <redacted>
---

[...]

Ok. Like in grep_submodule(), this should no longer add the submodule
ODB to the alternates list, so this call is now mostly used as a
fallback and also for testing.

To see if we are indeed testing this add-to-alternates case, I
reverted the change that made the code read from the submodule instead
of the_repository:

diff --git a/config.c b/config.c
index a85c12e6cc..cd37a9dcd9 100644
--- a/config.c
+++ b/config.c

@@ -1805,7 +1805,7 @@ int git_config_from_blob_oid(config_fn_t fn,
        unsigned long size;
        int ret;

-       buf = repo_read_object_file(repo, oid, &type, &size);
+       buf = read_object_file(oid, &type, &size);

Then, I ran t7814-grep-recurse-submodules.sh , where you've added the
GIT_TEST_FATAL_REGISTER_SUBMODULE_ODB=1 envvar. This correctly
produced the following error:

BUG: submodule.c:205: register_all_submodule_odb_as_alternates() called
[...]
not ok 23 - grep --recurse-submodules with submodules without
.gitmodules in the working tree

Nice! So the change made by this patch is covered by test 23. I think
it would be nice to mention that in this patch's message.