On Mon, Mar 16, 2020 at 04:51:12PM +0100, Miroslav Benes wrote:

quoted

diff --git a/arch/x86/xen/smp_pv.c b/arch/x86/xen/smp_pv.c
index 6b88cdcbef8f..39afd88309cb 100644
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c

@@ -92,6 +92,7 @@ asmlinkage __visible void cpu_bringup_and_idle(void)
 {
        cpu_bringup();
        boot_init_stack_canary();
+       asm volatile (UNWIND_HINT(ORC_REG_UNDEFINED, 0, ORC_TYPE_CALL, 1));
        cpu_startup_entry(CPUHP_AP_ONLINE_IDLE);
 }

and that seems to work. I need to properly verify and test, but the 
explanation is that as opposed to the above, cpu_startup_entry() is on the 
idle task's stack and the hint is then taken into account. The unwound 
stack seems to be complete, so it could indeed be the fix.

Not the correct one though. Objtool rightfully complains with

arch/x86/xen/smp_pv.o: warning: objtool: cpu_bringup_and_idle()+0x6a: undefined stack state

and all the other hacks I tried ended up in the same dead alley. It seems 
to me the correct fix is that all orc entries for cpu_bringup_and_idle() 
should have "end" property set to 1, since it is the first function on the 
stack. I don't know how to achieve that without the assembly hack in the 
patch I sent. If I am not missing something, of course.

Josh, any idea?

Yeah, I think mucking with the unwind hints in C code is going to be
precarious.  You could maybe have something like

	asm("
	  UNWIND_HINT_EMPTY\n
	  mov $CPUHP_AP_ONLINE_IDLE, %rdi\n
	  call cpu_startup_entry\n
	)"
	unreachable();

but that's pretty ugly (and it might not work anyway).

I suppose we could add a new facility to mark an entire C function as an
"end" point.

But I think it would be cleanest to just do something like your patch
and have the entry code be asm which then calls cpu_bringup_and_idle().
That would make it consistent with all other entry code, which all lives
in asm.

-- 
Josh