Re: Service lcores and Application lcores

From: Thomas Monjalon [mailto:thomas@monjalon.net]
Sent: Friday, June 30, 2017 10:29 AM
To: Van Haaren, Harry <redacted>
Cc: dev@dpdk.org; 'Jerin Jacob' <redacted>; Wiles, Keith
[off-list ref]; Richardson, Bruce [off-list ref]
Subject: Re: Service lcores and Application lcores

30/06/2017 10:52, Van Haaren, Harry:

quoted

From: Thomas Monjalon [mailto:thomas@monjalon.net]

quoted

29/06/2017 18:35, Van Haaren, Harry:

quoted

3) The problem;
   If a service core runs the SW PMD schedule() function (option 2) *AND*
   the application lcore runs schedule() func (option 1), the result is that
   two threads are concurrently running a multi-thread unsafe function.

Which function is multi-thread unsafe?

With the current design, the service-callback does not have to be multi-thread safe.
For example, the eventdev SW PMD is not multi-thread safe.

The service library handles serializing access to the service-callback if multiple cores
are mapped to that service. This keeps the atomic complexity in one place, and keeps
services as light-weight to implement as possible.

(We could consider forcing all service-callbacks to be multi-thread safe by using

atomics,

quoted

but we would not be able to optimize away the atomic cmpset if it is not required. This
feels heavy handed, and would cause useless atomic ops to execute.)

OK thank you for the detailed explanation.

quoted

quoted

Why the same function would be run by the service and by the scheduler?

The same function can be run concurrently by the application, and a service core.
The root cause that this could happen is that an application can *think* it is the
only one running threads, but in reality one or more service-cores may be running
in the background.

The service lcores and application lcores existence without knowledge of the others
behavior is the cause of concurrent running of the multi-thread unsafe service function.

That's the part I still don't understand.
Why an application would run a function on its own core if it is already
run as a service? Can we just have a check that the service API exists
and that the service is running?

The point is that really it is an application / service core mis-match.
The application should never run a PMD that it knows also has a service core running it.

However, porting applications to the service-core API has an over-lap time where an
application on 17.05 will be required to call eg: rte_eventdev_schedule() itself, and
depending on startup EAL flags for service-cores, it may-or-may-not have to call schedule() manually.

This is pretty error prone, and mis-configuration would cause A) deadlock due to no CPU cycles, B) segfault due to two cores.

As per the other thread re service-lcores[1], removing rte_eventdev_schedule() from the API would force apps to use the service-core infrastructure for eventdev instead of the possibility of mis-match.


[1] http://dpdk.org/ml/archives/dev/2017-June/069492.html