From: Junio C Hamano <hidden> Date: 2016-06-15 22:48:06
Jonathan Nieder [off-list ref] writes:
These patches are independent of the jn/makefile topic. I sent them
before, but I screwed up the sender so nobody received them. Anyway,
perhaps they could be useful.
I found all of them sensible but it seems to be based on version that is a
bit behind...
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:48:06
Junio C Hamano wrote:
Jonathan Nieder [off-list ref] writes:
quoted
These patches are independent of the jn/makefile topic. I sent them
before, but I screwed up the sender so nobody received them. Anyway,
perhaps they could be useful.
I found all of them sensible but it seems to be based on version that is a
bit behind...
Thanks. Patches 1-5 below are those I sent before, rebased against
maint. Since last time, remote-curl.o gained a few more dependencies,
which are included.
The remainder of this series is based on a merge of master and
patches 1-5:
Patch 6 adds another missing header dependency rule. It is not
included with the others because it applies to master, not maint.
Patches 7-12 revisit the topic of generating header dependency
rules on the fly. I updated them to produce patches 1-6, so I
thought, why not get some feedback on their present state?
Patch 7 clumps the header dependency rules together. I think this
makes them more readable; more importantly, later this will allow
ifdef'ing them out later in one fell swoop.
Patches 8-9 and 11 list all targets that can depend on headers in a
new OBJECTS variable, for use in patches 10 and 12.
Patch 10 generates header dependency rules on the fly. The resulting
dependency rules contain no useless dependencies (unlike the
hard-coded ones where almost everything depends on $(LIB_H)), so this
might be interesting to people wishing to speed up the
edit-compile-run cycle.
This only works with gcc. It should be possible to teach the relevant
options to clink.pl if there is demand, but not SCO cc.
Patch 12 is what I am most interested in: it uses the computed
dependencies to check that the hard-coded dependency rules are
correct. In the long term, maybe we will be able to get rid of the
hard-coded header dep rules altogether; in the short term, I am hoping
this checker can be useful for avoiding problems with the hard-coded
rules without spending too much human time on it.
Enjoy,
Jonathan Nieder (12):
Makefile: add missing header file dependencies
Makefile: make sure test helpers are rebuilt when headers change
Makefile: remove wt-status.h from LIB_H
Makefile: clean up http-walker.o dependency rules
Makefile: drop dependency on $(wildcard */*.h)
Makefile: transport.o depends on branch.h now
Makefile: rearrange dependency rules
Makefile: disable default implicit rules
Makefile: list generated object files in OBJECTS macro
Makefile: lazily compute header dependencies
Makefile: list standalone program object files in PROGRAM_OBJS
Teach Makefile to check header dependencies
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:48:06
LIB_H is missing exec_cmd.h and color.h. cache.h includes
SHA1_HEADER, and thus so does almost everything else, so add that
to LIB_H, too. xdiff-interface.h is not included by any header
files, but so many source files use xdiff that it is simplest to
include it in LIB_H, too.
xdiff-interface.o uses the xdiff library heavily; let it depend
on all xdiff headers to avoid needing to keep track of which
headers it uses.
Signed-off-by: Jonathan Nieder <redacted>
---
Makefile | 23 ++++++++++++++++++++---
1 files changed, 20 insertions(+), 3 deletions(-)
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:48:06
It is not worth the bother to maintain an up-to-date list of
which headers each test helper uses, so depend on $(LIB_H) to
catch them all.
Signed-off-by: Jonathan Nieder <redacted>
---
This applies to maint. Please let me know if you are interested
in the corresponding patch for master.
Makefile | 32 ++++++++++++++++----------------
1 files changed, 16 insertions(+), 16 deletions(-)
@@ -402,6 +402,18 @@ PROGRAMS += git-upload-pack$XPROGRAMS+=git-var$XPROGRAMS+=git-http-backend$X+TEST_PROGRAMS+=test-chmtime$X+TEST_PROGRAMS+=test-ctype$X+TEST_PROGRAMS+=test-date$X+TEST_PROGRAMS+=test-delta$X+TEST_PROGRAMS+=test-dump-cache-tree$X+TEST_PROGRAMS+=test-genrandom$X+TEST_PROGRAMS+=test-match-trees$X+TEST_PROGRAMS+=test-parse-options$X+TEST_PROGRAMS+=test-path-utils$X+TEST_PROGRAMS+=test-sha1$X+TEST_PROGRAMS+=test-sigchain$X+# List built-in command $C whose implementation cmd_$C() is not in# builtin-$C.o but is linked in as part of some other command.BUILT_INS+=$(patsubstbuiltin-%.o,git-%$X,$(BUILTIN_OBJS))
@@ -1732,18 +1746,6 @@ endif### Testing rules-TEST_PROGRAMS+=test-chmtime$X-TEST_PROGRAMS+=test-ctype$X-TEST_PROGRAMS+=test-date$X-TEST_PROGRAMS+=test-delta$X-TEST_PROGRAMS+=test-dump-cache-tree$X-TEST_PROGRAMS+=test-genrandom$X-TEST_PROGRAMS+=test-match-trees$X-TEST_PROGRAMS+=test-parse-options$X-TEST_PROGRAMS+=test-path-utils$X-TEST_PROGRAMS+=test-sha1$X-TEST_PROGRAMS+=test-sigchain$X-all::$(TEST_PROGRAMS)# GNU make supports exporting all variables by "export" without parameters.
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:48:06
A list of the few translation units using this header is
half-populated already. Including the dependency on this header
twice (once explicitly, once through LIB_H) makes it difficult to
figure out where future headers should be added to the Makefile.
Signed-off-by: Jonathan Nieder <redacted>
---
Makefile | 3 +--
1 files changed, 1 insertions(+), 2 deletions(-)
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:48:06
http-walker.o depends on http.h twice: once in the rule listing
files that use http.h, and again in the rule explaining how to
build it. Messy.
Signed-off-by: Jonathan Nieder <redacted>
---
Makefile | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:48:06
The files this pulls in are already pulled in by other dependency
rules (some recently added).
Signed-off-by: Jonathan Nieder <redacted>
---
Makefile | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:48:06
The git makefile never uses any default implicit rules.
Unfortunately, if a prerequisite for one of the intended rules is
missing, a default rule can be used in its place:
$ make var.s
CC var.s
$ rm var.c
$ make var.o
as -o var.o var.s
Avoiding the default rules avoids this hard-to-debug behavior.
It also should speed things up a little in the normal case.
Future patches may restrict the scope of the %.o: %.c pattern.
This patch would then ensure that for targets not listed, we do
not fall back to the default rule.
Signed-off-by: Jonathan Nieder <redacted>
---
Sadly, this does not really speed things up in the normal case.
I am not sure why, but make needs the -r option to avoid
considering the default rules, even though clearing the .SUFFIXES:
guarantees it will never use them.
Makefile | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:48:06
Set the OBJECTS variable to a comprehensive list of all object
file targets. To make sure it is truly comprehensive, restrict
the scope of the %.o pattern rule to only generate objects in
this list.
Attempts to build other object files will fail loudly:
$ touch foo.c
$ make foo.o
make: *** No rule to make target `foo.o'. Stop.
providing a reminder to add the new object to the OBJECTS list.
The new variable is otherwise unused. The intent is for later
patches to take advantage of it.
Signed-off-by: Jonathan Nieder <redacted>
---
Makefile | 9 +++++++--
1 files changed, 7 insertions(+), 2 deletions(-)
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:48:06
Use the gcc -MMD -MP -MF options to generate dependency rules as
a byproduct when building .o files if the
COMPUTE_HEADER_DEPENDENCIES variable is defined. That variable
is left undefined by default for now.
As each object file is built, write a makefile fragment
containing its dependencies in the deps/ subdirectory of its
containing directory. The deps/ directories should be generated
if they are missing at the start of each build. So let each
object file depend on $(missing_dep_dirs), which lists only the
directories of this kind that are missing to avoid needlessly
regenerating files when the directories' timestamps change.
gcc learned the -MMD -MP -MF options in version 3.0, so most gcc
users should have them by now.
The dependencies this option computes are more specific than the
rough estimates hard-coded in the Makefile, greatly speeding up
rebuilds when only a little-used header file has changed.
Signed-off-by: Jonathan Nieder <redacted>
---
.gitignore | 1 +
Makefile | 49 ++++++++++++++++++++++++++++++++++++++++++++-----
2 files changed, 45 insertions(+), 5 deletions(-)
@@ -217,6 +217,10 @@ all::# DEFAULT_EDITOR='~/bin/vi',# DEFAULT_EDITOR='$GIT_FALLBACK_EDITOR',# DEFAULT_EDITOR='"C:\Program Files\Vim\gvim.exe" --nofork'+#+# Define COMPUTE_HEADER_DEPENDENCIES if your compiler supports the -MMD option+# and you want to avoid rebuilding objects when an unrelated header file+# changes.GIT-VERSION-FILE:FORCE@$(SHELL_PATH)./GIT-VERSION-GEN
@@ -1677,14 +1681,48 @@ ASM_SRC := $(wildcard $(OBJECTS:o=S))ASM_OBJ:=$(ASM_SRC:S=o)C_OBJ:=$(filter-out$(ASM_OBJ),$(OBJECTS))+ifdef COMPUTE_HEADER_DEPENDENCIES+dep_dirs:=$(addsuffixdeps,$(sort$(dir$(OBJECTS))))+$(dep_dirs):+mkdir-p$@++missing_dep_dirs:=$(filter-out$(wildcard$(dep_dirs)),$(dep_dirs))+else+dep_dirs=+missing_dep_dirs=+endif+.SUFFIXES:-$(C_OBJ):%.o: %.cGIT-CFLAGS-$(QUIET_CC)$(CC)-o$*.o-c$(ALL_CFLAGS)$<+$(C_OBJ):%.o: %.cGIT-CFLAGS$(missing_dep_dirs)+$(QUIET_CC)$(CC)-o$*.o-c$(dep_args)$(ALL_CFLAGS)$<%.s:%.cGIT-CFLAGSFORCE$(QUIET_CC)$(CC)-S$(ALL_CFLAGS)$<-$(ASM_OBJ):%.o: %.SGIT-CFLAGS-$(QUIET_CC)$(CC)-o$*.o-c$(ALL_CFLAGS)$<+$(ASM_OBJ):%.o: %.SGIT-CFLAGS$(missing_dep_dirs)+$(QUIET_CC)$(CC)-o$*.o-c$(dep_args)$(ALL_CFLAGS)$<++ifdef COMPUTE_HEADER_DEPENDENCIES+# Take advantage of gcc's on-the-fly dependency generation+# See <http://gcc.gnu.org/gcc-3.0/features.html>.+dep_files:=$(wildcard$(foreachf,$(OBJECTS),$(dirf)deps/$(notdir$f).d))+ifneq ($(dep_files),)+include $(dep_files)+endif++dep_file=$(dir$@)deps/$(notdir$@).d+dep_args=-MF$(dep_file)-MMD-MP+else+dep_args=++# Dependencies on header files, for platforms that do not support+# the gcc -MMD option.+#+# Dependencies on automatically generated headers such as common-cmds.h+# should _not_ be included here, since they are necessary even when+# building an object for the first time.+#+# XXX. Please check occasionally that these include all dependencies+# gcc detects!$(GIT_OBJS):$(LIB_H)builtin-branch.o builtin-checkout.o builtin-clone.o builtin-reset.o branch.o transport.o:branch.h
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:48:06
Because of new commands like git-remote-http, the OBJECTS list
contains fictitious objects such as remote-http.o. Thus any
out-of-tree rules that require all $(OBJECTS) to be buildable
are broken. Add a list of real program objects to avoid this
problem.
Signed-off-by: Jonathan Nieder <redacted>
---
Preparing for patch 12.
Makefile | 24 +++++++++++++++---------
1 files changed, 15 insertions(+), 9 deletions(-)
@@ -390,12 +391,15 @@ EXTRA_PROGRAMS =# ... and all the rest that could be moved out of bindir to gitexecdirPROGRAMS+=$(EXTRA_PROGRAMS)-PROGRAMS+=git-fast-import$X-PROGRAMS+=git-imap-send$X-PROGRAMS+=git-shell$X-PROGRAMS+=git-show-index$X-PROGRAMS+=git-upload-pack$X-PROGRAMS+=git-http-backend$X++PROGRAM_OBJS+=fast-import.o+PROGRAM_OBJS+=imap-send.o+PROGRAM_OBJS+=shell.o+PROGRAM_OBJS+=show-index.o+PROGRAM_OBJS+=upload-pack.o+PROGRAM_OBJS+=http-backend.o++PROGRAMS+=$(patsubst%.o,git-%$X,$(PROGRAM_OBJS))TEST_PROGRAMS_NEED_X+=test-chmtimeTEST_PROGRAMS_NEED_X+=test-ctype
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:48:06
Put rules listing dependencies of compiled objects (.o files) on
header files (.h files) in one place, to make them easier to
compare and modify all at once.
Add a GIT_OBJS variable listing objects that depend on LIB_H,
for similar reasons.
No change in build-time behavior intended.
Signed-off-by: Jonathan Nieder <redacted>
---
Makefile | 49 +++++++++++++++++++++++++------------------------
1 files changed, 25 insertions(+), 24 deletions(-)
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:48:06
Add a target to use the gcc-generated makefile snippets for
dependencies on header files to check the hard-coded dependencies.
With this patch applied, if any dependencies are missing, then
make clean
make COMPUTE_HEADER_DEPENDENCIES=YesPlease
make CHECK_HEADER_DEPENDENCIES=YesPlease
will produce an error message like the following:
CHECK fast-import.o
missing dependencies: exec_cmd.h
make: *** [fast-import.o] Error 1
Signed-off-by: Jonathan Nieder <redacted>
---
That’s the end of the series. Thanks for reading.
Makefile | 95 ++++++++++++++++++++++++++++++++++++++++++++++++++++----------
1 files changed, 80 insertions(+), 15 deletions(-)
@@ -221,6 +221,9 @@ all::# Define COMPUTE_HEADER_DEPENDENCIES if your compiler supports the -MMD option# and you want to avoid rebuilding objects when an unrelated header file# changes.+#+# Define CHECK_HEADER_DEPENDENCIES to check for problems in the hard-coded+# dependency rules.GIT-VERSION-FILE:FORCE@$(SHELL_PATH)./GIT-VERSION-GEN
@@ -1693,33 +1702,89 @@ $(dep_dirs):mkdir-p$@missing_dep_dirs:=$(filter-out$(wildcard$(dep_dirs)),$(dep_dirs))-else+dep_file=$(dir$@)deps/$(notdir$@).d+dep_args=-MF$(dep_file)-MMD-MP+ifdef CHECK_HEADER_DEPENDENCIES+$(errorcannotcomputeheaderdependenciesoutsideanormalbuild. \+PleaseunsetCHECK_HEADER_DEPENDENCIESandtryagain)+endif+endif++ifndef COMPUTE_HEADER_DEPENDENCIES+ifndef CHECK_HEADER_DEPENDENCIESdep_dirs=missing_dep_dirs=+dep_args=+endif+endif++ifdef CHECK_HEADER_DEPENDENCIES+ifndef PRINT_HEADER_DEPENDENCIES+missing_deps=$(filter-out$(notdir$^),\+$(notdir$(shell$(MAKE)-s$@\+CHECK_HEADER_DEPENDENCIES=YesPlease\+USE_COMPUTED_HEADER_DEPENDENCIES=YesPlease\+PRINT_HEADER_DEPENDENCIES=YesPlease)))+endifendif+ASM_SRC:=$(wildcard$(OBJECTS:o=S))+ASM_OBJ:=$(ASM_SRC:S=o)+C_OBJ:=$(filter-out$(ASM_OBJ),$(OBJECTS))+.SUFFIXES:+ifdef PRINT_HEADER_DEPENDENCIES+$(C_OBJ):%.o: %.cFORCE+echo$^+$(ASM_OBJ):%.o: %.SFORCE+echo$^++ifndef CHECK_HEADER_DEPENDENCIES+$(errorcannotprintheaderdependenciesduringanormalbuild. \+PleasesetCHECK_HEADER_DEPENDENCIESandtryagain)+endif+endif++ifndef PRINT_HEADER_DEPENDENCIES+ifdef CHECK_HEADER_DEPENDENCIES+$(C_OBJ):%.o: %.c$(dep_files)FORCE+@set-e;echoCHECK$@;\+missing_deps="$(missing_deps)";\+ if test "$$missing_deps"; \+then\+echomissingdependencies:$$missing_deps;\+false;\+fi+$(ASM_OBJ):%.o: %.S$(dep_files)FORCE+@set-e;echoCHECK$@;\+missing_deps="$(missing_deps)";\+ if test "$$missing_deps"; \+then\+echomissingdependencies:$$missing_deps;\+false;\+fi+endif+endif++ifndef CHECK_HEADER_DEPENDENCIES$(C_OBJ):%.o: %.cGIT-CFLAGS$(missing_dep_dirs)$(QUIET_CC)$(CC)-o$*.o-c$(dep_args)$(ALL_CFLAGS)$<-%.s:%.cGIT-CFLAGSFORCE-$(QUIET_CC)$(CC)-S$(ALL_CFLAGS)$<$(ASM_OBJ):%.o: %.SGIT-CFLAGS$(missing_dep_dirs)$(QUIET_CC)$(CC)-o$*.o-c$(dep_args)$(ALL_CFLAGS)$<+endif-ifdef COMPUTE_HEADER_DEPENDENCIES+%.s:%.cGIT-CFLAGSFORCE+$(QUIET_CC)$(CC)-S$(ALL_CFLAGS)$<++ifdef USE_COMPUTED_HEADER_DEPENDENCIES# Take advantage of gcc's on-the-fly dependency generation# See <http://gcc.gnu.org/gcc-3.0/features.html>.-dep_files:=$(wildcard$(foreachf,$(OBJECTS),$(dirf)deps/$(notdir$f).d))-ifneq ($(dep_files),)-include $(dep_files)+dep_files_present:=$(wildcard$(dep_files))+ifneq ($(dep_files_present),)+include $(dep_files_present)endif--dep_file=$(dir$@)deps/$(notdir$@).d-dep_args=-MF$(dep_file)-MMD-MPelse-dep_args=-# Dependencies on header files, for platforms that do not support# the gcc -MMD option.#
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:48:06
Jonathan Nieder wrote:
The remainder of this series is based on a merge of master and
patches 1-5:
[...]
Jonathan Nieder (12):
Makefile: add missing header file dependencies
Makefile: make sure test helpers are rebuilt when headers change
Makefile: remove wt-status.h from LIB_H
Makefile: clean up http-walker.o dependency rules
Makefile: drop dependency on $(wildcard */*.h)
Makefile: transport.o depends on branch.h now
Makefile: rearrange dependency rules
Makefile: disable default implicit rules
Makefile: list generated object files in OBJECTS macro
Makefile: lazily compute header dependencies
Makefile: list standalone program object files in PROGRAM_OBJS
Teach Makefile to check header dependencies
For those would like to avoid redoing the conflict resolution (and
who wouldn’t?), this series is also available in the git repository
at
git://repo.or.cz/git/jrn.git autodep
If you’re interested, please take a look around and try it out. I’d
be happy to hear about any bugs you find.
Thanks again,
Jonathan
From: Jonathan Nieder <hidden> Date: 2016-06-15 22:48:06
Because of new commands like git-remote-http, the OBJECTS list
contains fictitious objects such as remote-http.o. Thus any
out-of-tree rules that require all $(OBJECTS) to be buildable
are broken. Add a list of real program objects to avoid this
problem.
To avoid duplication of effort, calculate the command list in
the PROGRAMS variable using the expansion of PROGRAM_OBJS.
This calculation occurs at the time $(PROGRAMS) is expanded,
so later additions to PROGRAM_OBJS will be reflected in it,
provided they occur before the build rules begin on line 1489.
Signed-off-by: Jonathan Nieder <redacted>
---
The previous version of this patch had a bug: I assumed that
PROGRAMS += $(patsubst %.o,git-%$X,$(PROGRAM_OBJS))
would add to PROGRAMS a list of command names derived from the value
of PROGRAM_OBJS at the time that line is first read; but in fact, what
matters is the value of PROGRAM_OBJS at expansion time.
More importantly, my process had a bug: I didn’t try 'make install'
before sending. Sorry about that. Here’s a fixed patch, at least.
I have pushed out this fix to
git://repo.or.cz/git/jrn.git autodep-rebased
git://repo.or.cz/git/jrn.git autodep
The former has been rerolled to include the fixed patch 11; the
latter includes a fixup commit at the end.
Thanks to Junio for the report.
Makefile | 28 ++++++++++++++++------------
1 files changed, 16 insertions(+), 12 deletions(-)
@@ -390,12 +391,15 @@ EXTRA_PROGRAMS =# ... and all the rest that could be moved out of bindir to gitexecdirPROGRAMS+=$(EXTRA_PROGRAMS)-PROGRAMS+=git-fast-import$X-PROGRAMS+=git-imap-send$X-PROGRAMS+=git-shell$X-PROGRAMS+=git-show-index$X-PROGRAMS+=git-upload-pack$X-PROGRAMS+=git-http-backend$X++PROGRAM_OBJS+=fast-import.o+PROGRAM_OBJS+=imap-send.o+PROGRAM_OBJS+=shell.o+PROGRAM_OBJS+=show-index.o+PROGRAM_OBJS+=upload-pack.o+PROGRAM_OBJS+=http-backend.o++PROGRAMS+=$(patsubst%.o,git-%$X,$(PROGRAM_OBJS))TEST_PROGRAMS_NEED_X+=test-chmtimeTEST_PROGRAMS_NEED_X+=test-ctype