RE: Re: [PATCH] net: ena: Add PCI shutdown handler to allow safe kexec

-----Original Message-----
From: netdev-owner@vger.kernel.org <redacted>
On Behalf Of David Miller [off-list ref]
Sent: Tuesday, March 24, 2020 6:05 AM
To: gpiccoli@canonical.com
Cc: netanel@amazon.com; akiyano@amazon.com; netdev@vger.kernel.org;
gtzalik@amazon.com; saeedb@amazon.com; zorik@amazon.com;
kernel@gpiccoli.net; gshan@redhat.com; gavin.guo@canonical.com;
jay.vosburgh@canonical.com; pedro.principeza@canonical.com
Subject: Re: [PATCH] net: ena: Add PCI shutdown handler to allow safe kexec

From: "Guilherme G. Piccoli" <redacted>
Date: Fri, 20 Mar 2020 09:55:34 -0300

quoted

Currently ENA only provides the PCI remove() handler, used during
rmmod for example. This is not called on shutdown/kexec path; we are
potentially creating a failure scenario on kexec:

(a) Kexec is triggered, no shutdown() / remove() handler is called for
ENA; instead pci_device_shutdown() clears the master bit of the PCI
device, stopping all DMA transactions;

(b) Kexec reboot happens and the device gets enabled again, likely
having its FW with that DMA transaction buffered; then it may trigger
the (now
invalid) memory operation in the new kernel, corrupting kernel memory

area.

quoted

This patch aims to prevent this, by implementing a shutdown() handler
quite similar to the remove() one - the difference being the handling
of the netdev, which is unregistered on remove(), but following the
convention observed in other drivers, it's only detached on shutdown().

This prevents an odd issue in AWS Nitro instances, in which after the
2nd kexec the next one will fail with an initrd corruption, caused by
a wild DMA write to invalid kernel memory. The lspci output for the
adapter present in my instance is:

00:05.0 Ethernet controller [0200]: Amazon.com, Inc. Elastic Network
Adapter (ENA) [1d0f:ec20]

Suggested-by: Gavin Shan <redacted>
Signed-off-by: Guilherme G. Piccoli <redacted>

Amazon folks, please review.

The patch is still under review we will reply as soon as we have finished testing it,
Thanks