Re: [UPDATED PATCH v3 10/52] arm, kvm: Fix CPU hotplug callback registration

From: Srivatsa S. Bhat <redacted>
Subject: [PATCH] arm, kvm: Fix CPU hotplug callback registration

Subsystems that want to register CPU hotplug callbacks, as well as perform
initialization for the CPUs that are already online, often do it as shown
below:

        get_online_cpus();

        for_each_online_cpu(cpu)
                init_cpu(cpu);

        register_cpu_notifier(&foobar_cpu_notifier);

        put_online_cpus();

This is wrong, since it is prone to ABBA deadlocks involving the
cpu_add_remove_lock and the cpu_hotplug.lock (when running concurrently
with CPU hotplug operations).

Instead, the correct and race-free way of performing the callback
registration is:

        cpu_notifier_register_begin();

        for_each_online_cpu(cpu)
                init_cpu(cpu);

        /* Note the use of the double underscored version of the API */
        __register_cpu_notifier(&foobar_cpu_notifier);

        cpu_notifier_register_done();


In the existing arm kvm code, there is no synchronization with CPU hotplug
to avoid missing the hotplug events that might occur after invoking
init_hyp_mode() and before calling register_cpu_notifier(). Fix this bug
and also use the new synchronization method (instead of get/put_online_cpus())
to ensure that we don't deadlock with CPU hotplug.

Cc: Gleb Natapov <gleb@kernel.org>
Cc: Russell King <redacted>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: kvmarm@lists.cs.columbia.edu
Cc: kvm@vger.kernel.org
Cc: linux-arm-kernel@lists.infradead.org
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Christoffer Dall <redacted>
Signed-off-by: Srivatsa S. Bhat <redacted>
---

 arch/arm/kvm/arm.c |    7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index bd18bb8..f0e50a0 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c

@@ -1051,21 +1051,26 @@ int kvm_arch_init(void *opaque)
                }
        }

+       cpu_notifier_register_begin();
+
        err = init_hyp_mode();
        if (err)
                goto out_err;

-       err = register_cpu_notifier(&hyp_init_cpu_nb);
+       err = __register_cpu_notifier(&hyp_init_cpu_nb);
        if (err) {
                kvm_err("Cannot register HYP init CPU notifier (%d)\n", err);
                goto out_err;
        }

+       cpu_notifier_register_done();
+
        hyp_cpu_pm_init();

        kvm_coproc_table_init();
        return 0;
 out_err:
+       cpu_notifier_register_done();
        return err;
 }