Re: [PATCH 3/7] eal: move virtual device probing into a bus

On Wed, Feb 15, 2017 at 02:27:47PM +0000, Shreyansh Jain wrote:

quoted

-----Original Message-----
From: Wiles, Keith [mailto:keith.wiles@intel.com]
Sent: Wednesday, February 15, 2017 7:53 PM
To: Shreyansh Jain <redacted>
Cc: Jan Blunck <redacted>; dev@dpdk.org
Subject: Re: [dpdk-dev] [PATCH 3/7] eal: move virtual device probing into a
bus

quoted

On Feb 15, 2017, at 8:15 AM, Shreyansh Jain [off-list ref] wrote:

On Wednesday 15 February 2017 07:41 PM, Shreyansh Jain wrote:

quoted

On Wednesday 15 February 2017 03:32 PM, Jan Blunck wrote:

quoted

This is a refactoring of the virtual device probing which moves into into
a proper bus structure.

Signed-off-by: Jan Blunck <redacted>
---
lib/librte_eal/common/eal_common_dev.c  | 22 -----------------
lib/librte_eal/common/eal_common_vdev.c | 44
+++++++++++++++++++++++++++++++++
2 files changed, 44 insertions(+), 22 deletions(-)

[...]

quoted

diff --git a/lib/librte_eal/common/eal_common_vdev.c

b/lib/librte_eal/common/eal_common_vdev.c
index 7d6e54f..523a3d6 100644

--- a/lib/librte_eal/common/eal_common_vdev.c
+++ b/lib/librte_eal/common/eal_common_vdev.c

@@ -37,8 +37,10 @@

#include <stdint.h>
#include <sys/queue.h>

[...]

quoted

+
+static struct rte_bus rte_vdev_bus = {
+    .scan = vdev_scan,
+    .probe = vdev_probe,
+};
+
+RTE_REGISTER_BUS_LATE(virtual, rte_vdev_bus);

Does it matter if VDEV buses are registered before or after other
buses? Either way, the callbacks would be called in the order specified
in EAL.

Just ignore this comment - I am misunderstood something.

But another question: Is there specific reason VDEV should be

registered/scanned *after* other devices? Is there some specific problem if
we do otherwise? (I think this is should be done, but I don't have a specific
reason).

Does the bonding driver which uses physical devices need to be registered
after physical ones? In Pktgen I noticed the vdev after the physical ports
and I could not blacklist them as the bonding driver needed them, which
caused the bonding ports to have a greater port number. In the case of pktgen
the bonding ports were up around 8 or 10 and caused the display to not show
the bonding ports. This is really just a usability problem for the developer
using Pktgen. I would like to see the vdev devices first, but as long as the
drivers (like bonding) are fine with them being first.

Ah, now I remember - there was a patch from Jerin for this.
Probably he is the best person to comment here.
(I don't have much insight here).

commit f4ce209a8ce5f416b61c76cee773bc54749e2048
Author: Jerin Jacob [off-list ref]
Date:   Sun Nov 20 13:30:50 2016 +0530

    eal: postpone vdev initialization

    Some platform like octeontx may use pci and
    vdev based combined device to represent a logical
    dpdk functional device.In such case, postponing the
    vdev initialization after pci device
    initialization will provide the better view of
    the pci device resources in the system in
    vdev's probe function, and it allows better
    functional subsystem registration in vdev probe
    function.

    As a bonus, This patch fixes a bond device
    initialization use case.

    example command to reproduce the issue:
    ./testpmd -c 0x2  --vdev 'eth_bond0,mode=0,
    slave=0000:02:00.0,slave=0000:03:00.0' --
    --port-topology=chained

    root cause:
    In existing case(vdev initialization and then pci
    initialization), creates three Ethernet ports with
    following port ids
    0 - Bond device
    1 - PCI device 0
    2 - PCI devive 1

    Since testpmd, calls the configure/start on all the ports on
    start up,it will translate to following illegal setup sequence

    1)bond device configure/start
    1.1) pci device0 stop/configure/start
    1.2) pci device1 stop/configure/start
    2)pci device 0 configure(illegal setup case,
    as device in start state)

    The fix changes the initialization sequence and
    allow initialization in following valid setup order
    1) pcie device 0 configure/start
    2) pcie device 1 configure/start
    3) bond device 2 configure/start
    3.1) pcie device 0/stop/configure/start
    3.2) pcie device 1/stop/configure/start