Re: [PATCH 0/6] examples/l3fwd-power: fix Rx interrupt mode for shared FD PMDs

On Fri, Sep 4, 2026 at 11:30 AM David Marchand
[off-list ref] wrote:

On Mon, 20 Jul 2026 at 10:01, Maxime Leroy [off-list ref] wrote:

quoted

l3fwd-power's Rx interrupt (one-shot) mode was written around NICs whose
interrupt fd is assigned per queue at setup, stays fixed and is
level-triggered, as on MSI-X hardware. PMDs that do not fit that model
never sleep on interrupts: they either fail to arm, watch an unbound fd,
or miss the wake, and the 10 ms poll timeout hides all of it.

dpaa2 is the motivating example: it delivers Rx interrupts through one
QBMan portal per lcore, so the queues of a lcore share a single fd that
is bound when the interrupt is enabled, not at queue setup; the
notification is edge-triggered on an empty to non-empty transition; and
arming a queue that already holds traffic returns -EAGAIN with the queue
left unarmed.

This series makes the interrupt and legacy loops handle those cases
while leaving MSI-X NICs unchanged:

  1/6 factor the duplicated arm/sleep/disarm block into a helper.
  2/6 check the arm return: on -EAGAIN poll and retry, on any other
      error drop interrupt mode for the lcore instead of sleeping on
      unarmed queues.
  3/6 register the fd in the epoll set after arming, not before, so a
      fd bound at enable time is watched only once it exists.
  4/6 accept -EEXIST when several queues share one fd.
  5/6 recheck the queues before sleeping so a packet that arrived during
      the arm window is not missed under edge-triggered interrupts.
  6/6 block indefinitely instead of on a 10 ms timeout, and wake the
      workers through an eventfd on exit, so a broken interrupt path no
      longer hides behind periodic polling.

Maxime Leroy (6):
  examples/l3fwd-power: factor out Rx interrupt sleep path
  examples/l3fwd-power: check Rx interrupt enable errors
  examples/l3fwd-power: enable Rx interrupt before epoll add
  examples/l3fwd-power: accept shared Rx interrupt FD
  examples/l3fwd-power: recheck Rx queues before sleeping
  examples/l3fwd-power: block until Rx interrupt or exit

 examples/l3fwd-power/main.c | 184 +++++++++++++++++++++++++++++-------
 1 file changed, 150 insertions(+), 34 deletions(-)

I had a quick look.
I am not a fan of changing applications because drivers behave
differently, but I think most of those changes are acceptable.

What differs here is not driver behaviour, it is where the interrupt fd
comes from.

On MSI-X NICs there is one fd per Rx queue, created at queue setup, and it
never changes. l3fwd-power is built on that: it adds every queue fd to its
epoll set once, before the loop, then only arms and sleeps.

On dpaa2 the interrupt arrives on the QBMan portal of the lcore, and a
portal has a single fd. So the fd belongs to the lcore, not to the queue,
and which fd a queue uses is decided when a lcore arms it, because that is
when the driver learns which lcore polls that queue.

3/6 and 4/6 are exactly these two consequences:

  - before the first arm, the queue has no fd yet, so the epoll
    registration has to happen after the arm, not before;
  - all the queues of one lcore share that single fd, so registering the
    second one returns -EEXIST, which the application treats as an error.

The ethdev API never promised one fd per queue, the application assumed it,
and no driver can fix that from its side since the epoll set belongs to the
application.

Where a driver can help, dpaa2 already does: it returns -EAGAIN when the
queue could not be armed, instead of pretending it was. That only works if
the caller checks, and today turn_on_off_intr() ignores the return value and
sleeps anyway. That is 2/6.

I sent one comment on the implementation of patch 3.

I will fix it in V2 thanks.

Patch 5 is a concern to me, as I don't think it solves anything, just
make the issue harder to reproduce maybe?

Patch 5 answered in its own thread.

Anatoly, Sivaprasad, please review.


--
David Marchand

--
Maxime Leroy