From: Junio C Hamano <hidden> Date: 2016-06-15 22:50:38
Jeff King [off-list ref] writes:
On Sat, Feb 19, 2011 at 05:11:03AM -0600, Jonathan Nieder wrote:
quoted
- merge builtins/ with commands/. It never was clear to me why
making a command builtin should require changing its filename.
This may be a bit too radical, but maybe the Makefile should use this
structure to save maintenance effort. In other words, is there any
reason not to just have:
BUILTIN_SOURCES = $(wildcard builtin/*.c)
BUILTIN_OBJS = $(patsubst builtin/%.c, builtin/%.o, $(BUILTIN_SOURCES))
and similar for LIB_OBJS and LIB_H?
The developers would have to be careful not to put a throw-away test
programs written in C (especially the ones that has its own "main()") in
builtin/ (or lib/) directories if we go that route. This obviously cuts
both ways; it sometimes is handy.
It is one less thing to need to do when writing new code, and one less
thing to have silly textual conflicts on. It probably doesn't matter
that much, though; we don't actually add new files or commands all that
often.
Yeah, you would need to add an entry to the builtin command list yourself
anyway, until we also autogenerate it, which I doubt will ever happen nor
is necessarily a good idea.
Speaking of Makefiles, one downside to all of this directory
segmentation is that you can't run "make" from the subdirectories.
I had an impression that "make -C lib/" would be one of the goals, iow,
when we split the directory structure, the next step would be to split the
top-level Makefile so that each directory is covered by its own Makefile,
just like Documentation/ is already usable that way.
Not that libgit.a in its current shape is very useful outside the context
of the git.git proper, though.
From: Jeff King <hidden> Date: 2016-06-15 22:50:38
On Tue, Feb 22, 2011 at 11:30:41AM -0800, Junio C Hamano wrote:
quoted
It is one less thing to need to do when writing new code, and one less
thing to have silly textual conflicts on. It probably doesn't matter
that much, though; we don't actually add new files or commands all that
often.
Yeah, you would need to add an entry to the builtin command list yourself
anyway, until we also autogenerate it, which I doubt will ever happen nor
is necessarily a good idea.
You also have to add the declaration to builtin.h. It is kind of a
hassle to have to add it in three places (Makefile, command list, and
declaration) in addition to the actual code. But again, it's not like we
do this that often.
quoted
Speaking of Makefiles, one downside to all of this directory
segmentation is that you can't run "make" from the subdirectories.
I had an impression that "make -C lib/" would be one of the goals, iow,
when we split the directory structure, the next step would be to split the
top-level Makefile so that each directory is covered by its own Makefile,
just like Documentation/ is already usable that way.
Ugh. I am not thrilled at the prospect of more recursive make. As it is
now, we have already dealt with bugs from the little bit of recursion
there is (e.g., propagating information between Makefiles) and weird
heisenbugs (I still occasionally get build failures on recursing into
the perl subdir during a parallel make). And then of course it's easy to
misuse; running "make" in t/ is not necessarily testing the latest
version. In my experience recursive make always comes with this sort of
flakiness.
Right now the actual C build process is entirely in the top-level
Makefile. I really like that I can trust it to handle dependencies
correctly, because it means I can bisect without resorting to "make
clean" for each step.
-Peff
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:50:38
Jeff King wrote:
On Tue, Feb 22, 2011 at 11:30:41AM -0800, Junio C Hamano wrote:
quoted
I had an impression that "make -C lib/" would be one of the goals, iow,
when we split the directory structure, the next step would be to split the
top-level Makefile so that each directory is covered by its own Makefile,
just like Documentation/ is already usable that way.
Ugh.
Two half-baked ideas:
A. top-level Makefile generates GIT-BUILD-OPTIONS and does not do
any work itself --- it just calls "make" in subdirectories to
do the real work. Other Makefiles slurp in
- GIT-BUILD-OPTIONS, to get the build options
- build-helpers/Makefile, for some pattern rules describing
how to build things
Benefit:
GIT-BUILD-OPTIONS becomes the normal way to pass build options,
so it is well tested and robust.
Downsides:
If the project grows too much, all those "make" calls start to
take time. It is possible for parallel "make" invocations to
trample on each other's work if they are not well coordinated.
B. top-level Makefile slurps in Makefiles from subdirs. Other
Makefiles would
- check if basic rules (pattern rules and GIT-BUILD-OPTIONS
generation) have been slurped in yet, and if not, include
them;
- check if build options have been slurped in yet, and
if not, include GIT-BUILD-OPTIONS;
- keep careful track of what directory "make" was run from; [*]
- define variables listing targets to be built in this
directory and rules describing how to build them.
Benefit:
GIT-BUILD-OPTIONS becomes the normal [...]. Single "make"
process, fast and predictable. "make" still works from
subdirectories.
Downside:
[*] is a little hazy and sounds hackish.
I admit that I have some irrational excitement about a lib/ that could
eventually be built and used separately from git. Mostly because I
think it could encourage people to make lib/ code comprehensible and
hard to misuse and to keep commands/ code minimal.
I really like that I can trust it to handle dependencies
correctly, because it means I can bisect without resorting to "make
clean" for each step.
Cross-directory dependencies have to be declared just as carefully as
dependencies within a directory. I wonder if there is any reason not
to make "make" under t/ imply "make -C .. all".
Sleepily,
Jonathan
From: Jeff King <hidden> Date: 2016-06-15 22:50:38
On Wed, Feb 23, 2011 at 02:29:29AM -0600, Jonathan Nieder wrote:
If the project grows too much, all those "make" calls start to
take time. It is possible for parallel "make" invocations to
trample on each other's work if they are not well coordinated.
Aren't there cases where no amount of coordination will help? E.g.,
let's say I have two subdirs, lib/ and cmds/. From the top-level I do
"make -j". This invokes two recursive sub-makes, one per directory. The
make in cmds/ sees that we haven't built stuff in lib/ yet. So it
invokes a recursive make in lib/. Now we have two parallel makes running
in lib/, stomping on each other. The problem is that no single make was
allowed to see the whole dependency tree.
I know GNU make does have some magic for communicating between recursive
makes. Does it handle this situation?
You can fix this with a rule like "only invoke recursive makes on
directories _below_ you, never above or to the side". But then you can't
run "make" from inside cmds/.
You could have a dependency in the top-level that says recursing into
cmds/ depends on having finished recursing into libs/. But that's not
strictly correct. You want to be working on part of what's in cmds/
(building .o files) in parallel with what's going on in lib/, and then
wait on lib/ only for the linking portion.
Those are all things that are trivial to handle in a single Makefile.
B. top-level Makefile slurps in Makefiles from subdirs. Other
Makefiles would
I like this better, but...
- keep careful track of what directory "make" was run from; [*]
[...]
[*] is a little hazy and sounds hackish.
Yeah, you have to be careful with paths. I think a more sane way would
be a single top-level Makefile that either contains everything, or
sources tidbits from subdirectories. But in either case, assumes it's
running from the top-level.
A dummy Makefile in each subdir that cd's to the toplevel and runs a
specific target. So from the top-level, "make" would build everything,
"make lib" would build stuff in the lib directory, and the Makefile in
lib/ would just do "cd .. && make lib".
It's not perfect, but it's simple and predictable.
-Peff
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:50:38
Jeff King wrote:
The problem is that no single make was
allowed to see the whole dependency tree.
I know GNU make does have some magic for communicating between recursive
makes. Does it handle this situation?
You can fix this with a rule like "only invoke recursive makes on
directories _below_ you, never above or to the side". But then you can't
run "make" from inside cmds/.
In that story, commands/Makefile would make commands/built-in.a and
various commands/foo.o. The toplevel makefile would pass "make -C
commands" a long list of targets to build, presumably deduced from
$(MAKECMDGOALS). Not pleasant, I agree.
quoted
- keep careful track of what directory "make" was run from; [*]
[...]
[*] is a little hazy and sounds hackish.
Yeah, you have to be careful with paths.
It could be pretty simple by maintaining a $(prefix_) variable
representing the path from the cwd to the directory containing the
makefile. Might try it out.
I think a more sane way would
be a single top-level Makefile that [...] contains everything
That sounds best to me in the short term, too.
A dummy Makefile in each subdir that cd's to the toplevel and runs a
specific target. So from the top-level, "make" would build everything,
"make lib" would build stuff in the lib directory, and the Makefile in
lib/ would just do "cd .. && make lib".