linux-next: build warnings after merge of the kbuild tree

Hi Michal,

After merging the kbuild tree, today's linux-next build (powerpc
ppc64_defconfig) produced these warnings:

WARNING: 25 bad relocations
c000000000cf2570 R_PPC64_ADDR64    __crc___arch_hweight16
c000000000cf2578 R_PPC64_ADDR64    __crc___arch_hweight32
c000000000cf2580 R_PPC64_ADDR64    __crc___arch_hweight64
c000000000cf2588 R_PPC64_ADDR64    __crc___arch_hweight8
c000000000cf2678 R_PPC64_ADDR64    __crc___bswapdi2
c000000000cf2690 R_PPC64_ADDR64    __crc___clear_user
c000000000cf26b8 R_PPC64_ADDR64    __crc___copy_tofrom_user
c000000000cf2728 R_PPC64_ADDR64    __crc___csum_partial
c000000000cf3f90 R_PPC64_ADDR64    __crc_copy_page
c000000000cf40e0 R_PPC64_ADDR64    __crc_csum_partial_copy_generic
c000000000cf4100 R_PPC64_ADDR64    __crc_current_stack_pointer
c000000000cf4928 R_PPC64_ADDR64    __crc_empty_zero_page
c000000000cf4db0 R_PPC64_ADDR64    __crc_flush_dcache_range
c000000000cf4dc0 R_PPC64_ADDR64    __crc_flush_icache_range
c000000000cf6470 R_PPC64_ADDR64    __crc_load_fp_state
c000000000cf6488 R_PPC64_ADDR64    __crc_load_vr_state
c000000000cf68d0 R_PPC64_ADDR64    __crc_memchr
c000000000cf68e0 R_PPC64_ADDR64    __crc_memcmp
c000000000cf68e8 R_PPC64_ADDR64    __crc_memcpy
c000000000cf6900 R_PPC64_ADDR64    __crc_memmove
c000000000cf6988 R_PPC64_ADDR64    __crc_memset
c000000000cf9328 R_PPC64_ADDR64    __crc_store_fp_state
c000000000cf9330 R_PPC64_ADDR64    __crc_store_vr_state
c000000000cf93d0 R_PPC64_ADDR64    __crc_strncmp
c000000000cf93d8 R_PPC64_ADDR64    __crc_strncpy

Introduced by commit

  9445aa1a3062 ("ppc: move exports to definitions")

I have reverted that commit for today.

[cc-ing the ppc guys for clues - also involved is commit

  22823ab419d8 ("EXPORT_SYMBOL() for asm")
]

-- 
Cheers,
Stephen Rothwell

On 2016-08-17 03:44, Stephen Rothwell wrote:

Hi Michal,

After merging the kbuild tree, today's linux-next build (powerpc
ppc64_defconfig) produced these warnings:

WARNING: 25 bad relocations
c000000000cf2570 R_PPC64_ADDR64    __crc___arch_hweight16

[...]

Introduced by commit

  9445aa1a3062 ("ppc: move exports to definitions")

I have reverted that commit for today.

[cc-ing the ppc guys for clues - also involved is commit

  22823ab419d8 ("EXPORT_SYMBOL() for asm")
]

FWIW, I see these warnings as well. Any help from ppc developers is
appreciated - should the R_PPC64_ADDR64 be whitelisted for exported asm
symbols (their CRCs actually)?

Thanks,
Michal

On Wed, 17 Aug 2016 14:59:59 +0200
Michal Marek [off-list ref] wrote:

On 2016-08-17 03:44, Stephen Rothwell wrote:

quoted

Hi Michal,

After merging the kbuild tree, today's linux-next build (powerpc
ppc64_defconfig) produced these warnings:

WARNING: 25 bad relocations
c000000000cf2570 R_PPC64_ADDR64    __crc___arch_hweight16  

[...]

quoted

Introduced by commit

  9445aa1a3062 ("ppc: move exports to definitions")

I have reverted that commit for today.

[cc-ing the ppc guys for clues - also involved is commit

  22823ab419d8 ("EXPORT_SYMBOL() for asm")
]  

FWIW, I see these warnings as well. Any help from ppc developers is
appreciated - should the R_PPC64_ADDR64 be whitelisted for exported asm
symbols (their CRCs actually)?

The dangling relocation is a side effect of linker unable to resolve the
reference to the undefined weak symbols. So the real question is, why has
genksyms not overridden these symbols with their CRC values?

This may not even be powerpc specific, but  I'll poke at it a bit more
when I get a chance.

Thanks,
Nick

Hi Nick,

On Thu, 18 Aug 2016 11:09:48 +1000 Nicholas Piggin [off-list ref] wrote:

On Wed, 17 Aug 2016 14:59:59 +0200
Michal Marek [off-list ref] wrote:

quoted

On 2016-08-17 03:44, Stephen Rothwell wrote:  

quoted

After merging the kbuild tree, today's linux-next build (powerpc
ppc64_defconfig) produced these warnings:

WARNING: 25 bad relocations
c000000000cf2570 R_PPC64_ADDR64    __crc___arch_hweight16    

[...]  

quoted

Introduced by commit

  9445aa1a3062 ("ppc: move exports to definitions")

I have reverted that commit for today.

[cc-ing the ppc guys for clues - also involved is commit

  22823ab419d8 ("EXPORT_SYMBOL() for asm")
]    

FWIW, I see these warnings as well. Any help from ppc developers is
appreciated - should the R_PPC64_ADDR64 be whitelisted for exported asm
symbols (their CRCs actually)?  

The dangling relocation is a side effect of linker unable to resolve the
reference to the undefined weak symbols. So the real question is, why has
genksyms not overridden these symbols with their CRC values?

This may not even be powerpc specific, but  I'll poke at it a bit more
when I get a chance.

Not sure if this is relevant, but with the commit reverted, the
__crc___... symbols are absolute.

00000000f55b3b3d A __crc___arch_hweight16

-- 
Cheers,
Stephen Rothwell

Hi Nick,

On Fri, 19 Aug 2016 13:38:54 +1000 Stephen Rothwell [off-list ref] wrote:

On Thu, 18 Aug 2016 11:09:48 +1000 Nicholas Piggin [off-list ref] wrote:

quoted

On Wed, 17 Aug 2016 14:59:59 +0200
Michal Marek [off-list ref] wrote:
  

quoted

On 2016-08-17 03:44, Stephen Rothwell wrote:    

quoted

After merging the kbuild tree, today's linux-next build (powerpc
ppc64_defconfig) produced these warnings:

WARNING: 25 bad relocations
c000000000cf2570 R_PPC64_ADDR64    __crc___arch_hweight16      

[...]    

quoted

Introduced by commit

  9445aa1a3062 ("ppc: move exports to definitions")

I have reverted that commit for today.

[cc-ing the ppc guys for clues - also involved is commit

  22823ab419d8 ("EXPORT_SYMBOL() for asm")
]      

FWIW, I see these warnings as well. Any help from ppc developers is
appreciated - should the R_PPC64_ADDR64 be whitelisted for exported asm
symbols (their CRCs actually)?    

The dangling relocation is a side effect of linker unable to resolve the
reference to the undefined weak symbols. So the real question is, why has
genksyms not overridden these symbols with their CRC values?

This may not even be powerpc specific, but  I'll poke at it a bit more
when I get a chance.  

Not sure if this is relevant, but with the commit reverted, the
__crc___... symbols are absolute.

00000000f55b3b3d A __crc___arch_hweight16

Ignore that :-)

I just had a look at a x86_64 allmodconfig result and it looks like the
weak symbols are not resolved their either ...

I may be missing something, but genksyms generates the crc's off the
preprocessed C source code and we don't have any for the asm files ...
-- 
Cheers,
Stephen Rothwell

On Fri, 19 Aug 2016 15:09:14 +1000
Stephen Rothwell [off-list ref] wrote:

Hi Nick,

On Fri, 19 Aug 2016 13:38:54 +1000 Stephen Rothwell [off-list ref] wrote:

quoted

On Thu, 18 Aug 2016 11:09:48 +1000 Nicholas Piggin [off-list ref] wrote:  

quoted

On Wed, 17 Aug 2016 14:59:59 +0200
Michal Marek [off-list ref] wrote:
    

quoted

On 2016-08-17 03:44, Stephen Rothwell wrote:      

quoted

After merging the kbuild tree, today's linux-next build (powerpc
ppc64_defconfig) produced these warnings:

WARNING: 25 bad relocations
c000000000cf2570 R_PPC64_ADDR64    __crc___arch_hweight16        

[...]      

quoted

Introduced by commit

  9445aa1a3062 ("ppc: move exports to definitions")

I have reverted that commit for today.

[cc-ing the ppc guys for clues - also involved is commit

  22823ab419d8 ("EXPORT_SYMBOL() for asm")
]        

FWIW, I see these warnings as well. Any help from ppc developers is
appreciated - should the R_PPC64_ADDR64 be whitelisted for exported asm
symbols (their CRCs actually)?      

The dangling relocation is a side effect of linker unable to resolve the
reference to the undefined weak symbols. So the real question is, why has
genksyms not overridden these symbols with their CRC values?

This may not even be powerpc specific, but  I'll poke at it a bit more
when I get a chance.    

Not sure if this is relevant, but with the commit reverted, the
__crc___... symbols are absolute.

00000000f55b3b3d A __crc___arch_hweight16  

Ignore that :-)

I just had a look at a x86_64 allmodconfig result and it looks like the
weak symbols are not resolved their either ...

I may be missing something, but genksyms generates the crc's off the
preprocessed C source code and we don't have any for the asm files ...

Looks like you're right, good find!

Thanks,
Nick

On 2016-08-19 07:09, Stephen Rothwell wrote:

Hi Nick,

On Fri, 19 Aug 2016 13:38:54 +1000 Stephen Rothwell [off-list ref] wrote:

quoted

On Thu, 18 Aug 2016 11:09:48 +1000 Nicholas Piggin [off-list ref] wrote:

quoted

On Wed, 17 Aug 2016 14:59:59 +0200
Michal Marek [off-list ref] wrote:
  

quoted

On 2016-08-17 03:44, Stephen Rothwell wrote:    

quoted

After merging the kbuild tree, today's linux-next build (powerpc
ppc64_defconfig) produced these warnings:

WARNING: 25 bad relocations
c000000000cf2570 R_PPC64_ADDR64    __crc___arch_hweight16      

[...]    

quoted

Introduced by commit

  9445aa1a3062 ("ppc: move exports to definitions")

I have reverted that commit for today.

[cc-ing the ppc guys for clues - also involved is commit

  22823ab419d8 ("EXPORT_SYMBOL() for asm")
]      

FWIW, I see these warnings as well. Any help from ppc developers is
appreciated - should the R_PPC64_ADDR64 be whitelisted for exported asm
symbols (their CRCs actually)?    

The dangling relocation is a side effect of linker unable to resolve the
reference to the undefined weak symbols. So the real question is, why has
genksyms not overridden these symbols with their CRC values?

This may not even be powerpc specific, but  I'll poke at it a bit more
when I get a chance.  

Not sure if this is relevant, but with the commit reverted, the
__crc___... symbols are absolute.

00000000f55b3b3d A __crc___arch_hweight16

Ignore that :-)

I just had a look at a x86_64 allmodconfig result and it looks like the
weak symbols are not resolved their either ...

I may be missing something, but genksyms generates the crc's off the
preprocessed C source code and we don't have any for the asm files ...

Of course you are right. Which means that we are losing type information
for these exports for CONFIG_MODVERSIONS purposes. I guess it's
acceptable, since the asm functions are pretty basic and their
signatures do not change.

Michal

On Fri, 19 Aug 2016 10:37:00 +0200
Michal Marek [off-list ref] wrote:

On 2016-08-19 07:09, Stephen Rothwell wrote:

quoted

Hi Nick,

On Fri, 19 Aug 2016 13:38:54 +1000 Stephen Rothwell [off-list ref] wrote:  

quoted

On Thu, 18 Aug 2016 11:09:48 +1000 Nicholas Piggin [off-list ref] wrote:  

quoted

On Wed, 17 Aug 2016 14:59:59 +0200
Michal Marek [off-list ref] wrote:
    

quoted

On 2016-08-17 03:44, Stephen Rothwell wrote:      

quoted

After merging the kbuild tree, today's linux-next build (powerpc
ppc64_defconfig) produced these warnings:

WARNING: 25 bad relocations
c000000000cf2570 R_PPC64_ADDR64    __crc___arch_hweight16        

[...]      

quoted

Introduced by commit

  9445aa1a3062 ("ppc: move exports to definitions")

I have reverted that commit for today.

[cc-ing the ppc guys for clues - also involved is commit

  22823ab419d8 ("EXPORT_SYMBOL() for asm")
]        

FWIW, I see these warnings as well. Any help from ppc developers is
appreciated - should the R_PPC64_ADDR64 be whitelisted for exported asm
symbols (their CRCs actually)?      

The dangling relocation is a side effect of linker unable to resolve the
reference to the undefined weak symbols. So the real question is, why has
genksyms not overridden these symbols with their CRC values?

This may not even be powerpc specific, but  I'll poke at it a bit more
when I get a chance.    

Not sure if this is relevant, but with the commit reverted, the
__crc___... symbols are absolute.

00000000f55b3b3d A __crc___arch_hweight16  

Ignore that :-)

I just had a look at a x86_64 allmodconfig result and it looks like the
weak symbols are not resolved their either ...

I may be missing something, but genksyms generates the crc's off the
preprocessed C source code and we don't have any for the asm files ...  

Of course you are right. Which means that we are losing type information
for these exports for CONFIG_MODVERSIONS purposes. I guess it's
acceptable, since the asm functions are pretty basic and their
signatures do not change.

I don't completely agree. It would be nice to have the functionality
still there.

What happens if you just run cmd_modversions on the as rule? It relies on
!defined(__ASSEMBLY__), but we're feeding the result to genksyms, not as.
It would require the header be included in the .S file and be protected for
asm builds.

Stephen wasn't a fan of suck a hack and I can't say I blame him. Another
possibility I suppose is an EXPORT_SYMBOL_ASM() variant that takes string
containing C function declaration and just inserts it as an assembler
comment somewhere that genksysms can find.


Thanks,
Nick

On Fri, 19 Aug 2016 20:44:55 +1000
Nicholas Piggin [off-list ref] wrote:

On Fri, 19 Aug 2016 10:37:00 +0200
Michal Marek [off-list ref] wrote:

quoted

On 2016-08-19 07:09, Stephen Rothwell wrote:  

[snip]

quoted

quoted

I may be missing something, but genksyms generates the crc's off the
preprocessed C source code and we don't have any for the asm files ...    

Of course you are right. Which means that we are losing type information
for these exports for CONFIG_MODVERSIONS purposes. I guess it's
acceptable, since the asm functions are pretty basic and their
signatures do not change.  

I don't completely agree. It would be nice to have the functionality
still there.

What happens if you just run cmd_modversions on the as rule? It relies on
!defined(__ASSEMBLY__), but we're feeding the result to genksyms, not as.
It would require the header be included in the .S file and be protected for
asm builds.

This seems like it *could* be made to work, but there's a few problems.

- .h files are not made for C consumption. Matter of manually adding the
ifdef guards, which isn't terrible.

- .S files do not all include their .h where the C declaration is. Also
will cause some churn but doable and maybe not completely unreasonable.

- genksyms parser barfs when it hits the assembly of the .S file. Best
way to fix that seems just send the #include and EXPORT_SYMBOL lines
from the .S to the preprocessor. That's a bit of a rabbit hole too, with
some .S files being included, etc.

I'm not sure what to do here. If nobody cares and we lose CRCs for .S
exports, then okay we can whitelist those relocs easily. If we don't want
to lose the functionality, the above might work but it's a bit intrusive
an is going to require another cycle of prep patches to go through arch
code first.

Or suggestions for alternative approach?

Thanks,
Nick

On Mon, 22 Aug 2016 20:47:58 +1000
Nicholas Piggin [off-list ref] wrote:

On Fri, 19 Aug 2016 20:44:55 +1000
Nicholas Piggin [off-list ref] wrote:

quoted

On Fri, 19 Aug 2016 10:37:00 +0200
Michal Marek [off-list ref] wrote:
  

quoted

On 2016-08-19 07:09, Stephen Rothwell wrote:    

[snip]

quoted

quoted

quoted

I may be missing something, but genksyms generates the crc's off the
preprocessed C source code and we don't have any for the asm files ...      

Of course you are right. Which means that we are losing type information
for these exports for CONFIG_MODVERSIONS purposes. I guess it's
acceptable, since the asm functions are pretty basic and their
signatures do not change.    

I don't completely agree. It would be nice to have the functionality
still there.

What happens if you just run cmd_modversions on the as rule? It relies on
!defined(__ASSEMBLY__), but we're feeding the result to genksyms, not as.
It would require the header be included in the .S file and be protected for
asm builds.  

This seems like it *could* be made to work, but there's a few problems.

- .h files are not made for C consumption. Matter of manually adding the
ifdef guards, which isn't terrible.

- .S files do not all include their .h where the C declaration is. Also
will cause some churn but doable and maybe not completely unreasonable.

- genksyms parser barfs when it hits the assembly of the .S file. Best
way to fix that seems just send the #include and EXPORT_SYMBOL lines
from the .S to the preprocessor. That's a bit of a rabbit hole too, with
some .S files being included, etc.

I'm not sure what to do here. If nobody cares and we lose CRCs for .S
exports, then okay we can whitelist those relocs easily. If we don't want
to lose the functionality, the above might work but it's a bit intrusive
an is going to require another cycle of prep patches to go through arch
code first.

Or suggestions for alternative approach?

Here is a quick patch that I think should catch missing CRCs in
architecture independent way. If we merge something like this, we
can whitelist the symbols in arch/powerpc so people get steered to
the right place.

Powerpc seems to be the only one really catching this, and it's
only as a side effect of a test run for CONFIG_RELOCATABLE kernels,
which means version failures probably slipped through other archs.

I'll clean it up, do some more testing, and submit it unless
anybody dislikes it or has a better way to do it.

Thanks,
Nick

diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index 4b8ffd3..1efc454 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c

@@ -609,6 +609,7 @@ static void handle_modversions(struct module *mod, struct elf_info *info,
 {
 	unsigned int crc;
 	enum export export;
+	int is_crc = 0;
 
 	if ((!is_vmlinux(mod->name) || mod->is_dot_o) &&
 	    strncmp(symname, "__ksymtab", 9) == 0)

@@ -618,6 +619,7 @@ static void handle_modversions(struct module *mod, struct elf_info *info,
 
 	/* CRC'd symbol */
 	if (strncmp(symname, CRC_PFX, strlen(CRC_PFX)) == 0) {
+		is_crc = 1;
 		crc = (unsigned int) sym->st_value;
 		sym_update_crc(symname + strlen(CRC_PFX), mod, crc,
 				export);

@@ -663,6 +665,10 @@ static void handle_modversions(struct module *mod, struct elf_info *info,
 		else
 			symname++;
 #endif
+		if (is_crc && !mod->is_dot_o) {
+			const char *e = is_vmlinux(mod->name) ?"":".ko";
+			warn("EXPORT symbol \"%s\" [%s%s] version generation failed, symbol will not be versioned.\n", symname + strlen(CRC_PFX), mod->name, e);
+		}
 		mod->unres = alloc_symbol(symname,
 					  ELF_ST_BIND(sym->st_info) == STB_WEAK,
 					  mod->unres);

On Fri, Aug 26, 2016 at 01:58:03PM +1000, Nicholas Piggin wrote:

should that not be a bool here ?

is_crc = true;

 		crc = (unsigned int) sym->st_value;
 		sym_update_crc(symname + strlen(CRC_PFX), mod, crc,
 				export);

thx!
hofrat