Re: [PATCH v4 02/17] octeontx2-pf: Mailbox communication with AF

On Tue, Jan 21, 2020 at 9:30 PM Jakub Kicinski [off-list ref] wrote:

On Tue, 21 Jan 2020 18:51:36 +0530, sunil.kovvuri@gmail.com wrote:

quoted

From: Sunil Goutham <sgoutham@marvell.com>

In the resource virtualization unit (RVU) each of the PF and AF
(admin function) share a 64KB of reserved memory region for
communication. This patch initializes PF <=> AF mailbox IRQs,
registers handlers for processing these communication messages.
Also adds support to process these messages in both directions
ie responses to PF initiated DOWN (PF => AF) messages and AF
initiated UP messages (AF => PF).

Mbox communication APIs and message formats are defined in AF driver
(drivers/net/ethernet/marvell/octeontx2/af), mbox.h from AF driver is
included here to avoid duplication.

Signed-off-by: Geetha sowjanya <gakula@marvell.com>
Signed-off-by: Christina Jacob <redacted>
Signed-off-by: Subbaraya Sundeep <sbhatta@marvell.com>
Signed-off-by: Aleksey Makarov <redacted>
Signed-off-by: Sunil Goutham <sgoutham@marvell.com>

quoted

 struct otx2_hw {
+
+     /* MSI-X*/

                ^

The white space here is fairly loose

Will fix the white space issues.

quoted

+static inline void otx2_sync_mbox_bbuf(struct otx2_mbox *mbox, int devid)
+{
+     u16 msgs_offset = ALIGN(sizeof(struct mbox_hdr), MBOX_MSG_ALIGN);
+     void *hw_mbase = mbox->hwbase + (devid * MBOX_SIZE);
+     struct otx2_mbox_dev *mdev = &mbox->dev[devid];
+     struct mbox_hdr *hdr;
+     u64 msg_size;
+
+     if (mdev->mbase == hw_mbase)
+             return;
+
+     hdr = hw_mbase + mbox->rx_start;
+     msg_size = hdr->msg_size;
+
+     if (msg_size > mbox->rx_size - msgs_offset)
+             msg_size = mbox->rx_size - msgs_offset;
+
+     /* Copy mbox messages from mbox memory to bounce buffer */
+     memcpy(mdev->mbase + mbox->rx_start,
+            hw_mbase + mbox->rx_start, msg_size + msgs_offset);

I'm slightly concerned about the use of non-iomem helpers like memset
and memcpy on what I understand to be IOMEM, and the lack of memory
barriers. But then again, I don't know much about iomem_wc(), is this
code definitely correct from memory ordering perspective?
(The memory barrier in otx2_mbox_msg_send() should probably be just
wmb(), syncing with HW is unrelated with SMP.)

The mailbox region is a normal memory which is exposed to two devices
via PCI BARs.
And the writeq() call (to trigger IRQ) inside otx2_mbox_msg_send() has a wmb().

Thanks,
Sunil.