From: Jerin Jacob [mailto:jerin.jacob@caviumnetworks.com]
Sent: Friday, August 10, 2018 3:52 PM
To: Elo, Matias (Nokia - FI/Espoo) <redacted>
Cc: Van Haaren, Harry <redacted>; dev@dpdk.org
Subject: Re: [dpdk-dev] eventdev: method for finding out unlink status

-----Original Message-----

quoted

Date: Fri, 10 Aug 2018 14:24:02 +0000
From: "Elo, Matias (Nokia - FI/Espoo)" <redacted>
To: Jerin Jacob <redacted>
CC: "Van Haaren, Harry" <redacted>, "dev@dpdk.org"
 [off-list ref]
Subject: Re: [dpdk-dev] eventdev: method for finding out unlink status
x-mailer: Apple Mail (2.3445.9.1)

quoted

# Other than that, I am still not able to understand, why not
application wait until rte_event_port_unlink() returns.

Making rte_event_port_unlink() blocking would be troublesome if one

doesn’t care

quoted

about unlink completion. E.g. doing dynamic load balancing.

By making it as blocking(i.e the rte_event_port_unlink() returns when
unlink() completed) forcing everyone to care about unlink completion.
Right?

I'm not sure I understand the issue here.
Is anybody suggesting to make unlink() blocking?

For certain PMDs, perhaps it must be a synchronous handled unlink().
For other PMDs (eg event/sw) there are multiple threads involved,
so it must be async. Hence, APIs should be async to avoid blocking the caller.

With an async API, if you don't want the async behaviuor, it is
easy to build the sync version: call it in a loop, optionally with a delay().

quoted

quoted

# What in real word use case, application can, do other than waiting
to complete rte_event_port_unlink(). If we try to put some logic in

like,

quoted

quoted

while (rte_event_port_unlink_in_progress(dev, port) > 0){
      do_something();
}

The do_something() will not be called in some platform at all.

# Any idea on what will be the real world use case, where

rte_event_port_unlink() called in fastpath?

quoted

In our application this could be used for example to pause scheduling of

new events while

quoted

working on an “expensive” event to minimise delays. It is also needed when

destroying

quoted

queues, though calling this fast path is debatable (our application

enables creating /

quoted

destroying queues at runtime).

If I understand it correctly, Your current issue is, SW driver is
not waiting for to complete the unlink() operation so that in your
application you are seeing some abnormalities.

To be more specific:

The issue is that the application cannot reliably know when the unlink()
has completed. As such, the application doesn't know when it can put
a core to sleep instead of busy polling.

Waiting for unlink() to return does not give this info - the
scheduler core may schedule more events until it "acks" the unlink.

Checking that dequeue() == 0 events isn't reliable either, the buffering
in the PMD could hide events.

It seems logical to add an API that allows the user to query the status
of the PMDs unlink() progress. This provides the application with the
information that it needs to reliably determine when a worker is no
longer being scheduled events of a specific type, at which point it
can put its worker to sleep.

quoted

These are perhaps not the best examples but I would be very cautious to

make a function

quoted

blocking if there is even a small probability that it could be called from

the fast path.

Let assume even if it is called in fastpath, what else, we can really do
other that calling rte_pause() in loop. realistically? after issuing
unlink() operation.

A CPU core could be polling two queues:

for_each(queue) {
  if (unlinks_in_progress() != 0)
    continue;
  // poll queue and handle packets
}

or we could have the unlinks in progress flush buffers:

if (unlinks_in_progress() != 0) {
  tx_flush_packets();
}

I don't think that making unlinks_in_progress() blocking is acceptable,
it feels like too strong a limitation.

If a PMD doesn't support unlinking, we can have the eventdev.c layer
return an error code (or zero..?) to avoid burden on PMDs that don't care.


I've detailed why I think overloading the return value of unlink()
(eg to -EBUSY) isn't a good solution here:
http://mails.dpdk.org/archives/dev/2018-August/109550.html