RE: Re: [PATCH] net: ena: Add PCI shutdown handler to allow safe kexec

-----Original Message-----
From: Jubran, Samih
Sent: Tuesday, March 24, 2020 7:03 PM
To: 'David Miller' <davem@davemloft.net>; 'gpiccoli@canonical.com'
[off-list ref]
Cc: Belgazal, Netanel <redacted>; Kiyanovski, Arthur
[off-list ref]; 'netdev@vger.kernel.org'
[off-list ref]; Tzalik, Guy [off-list ref]; Bshara,
Saeed [off-list ref]; Machulsky, Zorik [off-list ref];
'kernel@gpiccoli.net' [off-list ref]; 'gshan@redhat.com'
[off-list ref]; 'gavin.guo@canonical.com'
[off-list ref]; 'jay.vosburgh@canonical.com'
[off-list ref]; 'pedro.principeza@canonical.com'
[off-list ref]
Subject: RE: Re: [PATCH] net: ena: Add PCI shutdown handler to allow safe
kexec

quoted

-----Original Message-----
From: netdev-owner@vger.kernel.org <redacted>

On

quoted

Behalf Of David Miller [off-list ref]
Sent: Tuesday, March 24, 2020 6:05 AM
To: gpiccoli@canonical.com
Cc: netanel@amazon.com; akiyano@amazon.com;

netdev@vger.kernel.org;

quoted

gtzalik@amazon.com; saeedb@amazon.com; zorik@amazon.com;
kernel@gpiccoli.net; gshan@redhat.com; gavin.guo@canonical.com;
jay.vosburgh@canonical.com; pedro.principeza@canonical.com
Subject: Re: [PATCH] net: ena: Add PCI shutdown handler to allow safe
kexec

From: "Guilherme G. Piccoli" <redacted>
Date: Fri, 20 Mar 2020 09:55:34 -0300

quoted

Currently ENA only provides the PCI remove() handler, used during
rmmod for example. This is not called on shutdown/kexec path; we are
potentially creating a failure scenario on kexec:

(a) Kexec is triggered, no shutdown() / remove() handler is called
for ENA; instead pci_device_shutdown() clears the master bit of the
PCI device, stopping all DMA transactions;

(b) Kexec reboot happens and the device gets enabled again, likely
having its FW with that DMA transaction buffered; then it may
trigger the (now
invalid) memory operation in the new kernel, corrupting kernel
memory

area.

quoted

This patch aims to prevent this, by implementing a shutdown()
handler quite similar to the remove() one - the difference being the
handling of the netdev, which is unregistered on remove(), but
following the convention observed in other drivers, it's only detached on

shutdown().

quoted

quoted

This prevents an odd issue in AWS Nitro instances, in which after
the 2nd kexec the next one will fail with an initrd corruption,
caused by a wild DMA write to invalid kernel memory. The lspci
output for the adapter present in my instance is:

00:05.0 Ethernet controller [0200]: Amazon.com, Inc. Elastic Network
Adapter (ENA) [1d0f:ec20]

Suggested-by: Gavin Shan <redacted>
Signed-off-by: Guilherme G. Piccoli <redacted>

Amazon folks, please review.

The patch is still under review we will reply as soon as we have finished
testing it, Thanks

Acked-by: Sameeh Jubran <redacted>