On Thu, Aug 30, 2018 at 7:37 PM Arnd Bergmann [off-list ref] wrote:

On Tue, Aug 28, 2018 at 3:10 PM Sunil Kovvuri [off-list ref] wrote:

quoted

quoted

quoted

quoted

If this is a regular PCI ethernet driver, why do you put it into driver/soc
rather than drivers/net/ethernet/ ?

No, this is not a ethernet driver, as mentioned in the cover letter
this driver and AF driver doesn't
handle any IO. There will be a separate ethernet driver (will submit
that as well in future) which will
communicate with these drivers for configuring hardware.

The driver in question here is for a serdes controller which handles
physical ethernet interfaces.
Admin function driver gathers info w.r.t current state of physical
ethernet interfaces from this driver
and notifies actual ethernet driver about changes, if any.

Ok. Can you describe the structure that the PCI devices appear
in? It might help to be make the connection between the differnet
patches to understand how things fit together. In the final
picture, how many different pci_driver instances do you have,
and what part are they for?

List of PCI devices are CGX, RVU PF0-PFn SRIOV physical functions
and RVU VF0-VFn SRIOV virtual functions. No of VFs per PF is configurable
and done by low level firmware.

List of PCI driver instances would be CGX driver, RVU PF0 (i.e admin
function) driver,
PF1-PFn either netdev driver or crypto driver, VF0-VFn functionality would be
same as their PF.

The current plan is to have CGX driver, Admin function driver, PF
netdev driver, VF netdev driver and PF/VF crypto drivers.

Ok, I think I understand the PF/VF distinction now. One (to me)
surprising aspect here is that you not just have one physical function
that you can use to assign resources to multiple virtual functions,
but also a second level of virtualization that is used to assign
resources to "physical functions" that are less physical than the
name suggests.

Yes, PF is just for name sake, on-boot there is no difference between
PFs/VFs as such.
PF0 has privilege access to assign resources to all PFs and their VFs.
This admin function driver loads for PF0.

The part that I have not grasped yet is what the split between
the CGX and the AF is for, how they relate to one another, and
what the software abstraction for the two is going to be.

In HW, CGX is a separate PCI device which handles the serdes and
physical ethernet interface.
Ethernet driver in drivers/net/ethernet can only communicate to
admin function driver since they share a mailbox memory.
So we had to bind both CGX and admin function drivers to almost work as one,
inorder to provide relavent info to ethernet drivers. That's why we
have many functions
from CGX driver which AF uses.

eg: Firmware gets to know about a physical interface status change,
which CGX driver gets
to know and it uses AF's mailbox communication to inform ethernet
driver about the event.

quoted

quoted

Is the idea that an ethernet device driver always attaches to a
virtual function that gets created by the main driver, and that
the two drivers share no interfaces on the kernel side, or do
you have multiple drivers linking to each other?

Ethernet device driver can attach to both physical function and virtual function
whose HW resources are provisioned by admin function driver.

Yes the PF/VF ethernet drivers and these drivers won't share any
kernel interfaces.
Physical ethernet interface is owned by ethernet driver only, this driver just
configures which ethernet driver instance uses which physcial interface.

Ok.

         Arnd