Re: [PATCH v3 7/8] mk: sort object files when building deps lists

27/06/2017 16:51, Luca Boccassi:

On Tue, 2017-06-27 at 15:52 +0200, Thomas Monjalon wrote:

quoted

27/06/2017 12:43, Luca Boccassi:

quoted

On Tue, 2017-06-27 at 01:20 +0200, Thomas Monjalon wrote:

quoted

23/06/2017 20:41, lboccass@brocade.com:

quoted

From: Luca Boccassi <redacted>

In order to achieve reproducible builds, always use the same
order when listing object files to build dependencies lists.

Signed-off-by: Luca Boccassi <redacted>
---
 mk/rte.app.mk     | 4 ++--
 mk/rte.hostapp.mk | 4 ++--
 mk/rte.shared.mk  | 4 ++--
 3 files changed, 6 insertions(+), 6 deletions(-)

--- a/mk/rte.app.mk
+++ b/mk/rte.app.mk

@@ -263,8 +263,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-

l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
 
 # list of found libraries files (useful for deps). If not
found,
the
 # library is silently ignored and dep won't be checked
-LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
-	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
+LDLIBS_FILES := $(sort $(wildcard $(foreach
dir,$(LDLIBS_PATH),\
+	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))

You cannot sort libraries.
Check - for instance - this comment above in this file:
	# Eliminate duplicates without sorting, only keep the last
occurrence
	filter-libs = \

Not sure I follow - what's the reason for avoiding to sort the list
of
libs to link against?

Sorry, the ordering issue is with LDLIBS not LDLIBS_FILES.

quoted

quoted

Why sorting them?
What is random in libraries list?

The issue is that the output of wildcard is not fully
deterministic. It
can depend on the filesystem, and even on the locale settings [1].
Before GNU Make 3.82 (2009) it used to automatically sort the
output,
but that was removed (to make it faster... sigh). [2]

It is not a true wildcard here. It is just filtering files which
do not exist.
I think you do not need this patch for deterministic build.

But then those lists are passed down in the .SECONDEXPANSION rule
right?

I do not follow you.
Please explain what is the benefit of the patch in the next version.

I am trying to find out an alternative solution. The problem to solve
is that the build system picks the public headers path (which is
embedded in the object files as notation and in the debug symbol) from
a seemingly random location between the make install path and the
source path (build/include/rte_foo.h vs lib/librte_foo/rte_foo.h) and
this makes the build not reproducible.

Nonetheless, while I work more on the last 4 patches, could you please
have a look and eventually take patch 3 and 4? They are needed to
respectively have a deterministic list of files in the libdpdk.so
linker script and a list of source files in one of the example
documentation files.

I think it's better to consider and apply the whole series making
a reproducible build.
The rule is to avoid splitting series without good reason,
so tracking patches is easier.