On Mon 2023-01-09 16:12:53, Luis Chamberlain wrote:

On Mon, Jan 09, 2023 at 02:40:27PM +0100, Petr Mladek wrote:

quoted

Why are try hardly comparable?

1. The speed depends on the number of loaded modules
   and number of symbols. It highly depends on the configuration
   that was used to build the kernel.

2. The test runs only once. As a result it is hard to judge
   how big is the noise.

3. The noise might depend on the size and state of CPU caches.


I personally vote for removing this selftest!

Even so, just as with testing a filesystem with different types of
configurations, at least testing a few configs helps and it's what
we do. Then, if anyone ever wanted to try to increase performance
on symbol lookup today they have no easy way to measure things. How
would they go about comparing things performance without this selftest?

How many people cares about kallsyms performance, please?
Is it worth spending time one implementing and maintaining such a
selftest?

Yes, Zhen wanted to make it faster. But how likely will anyone else
try to make it even better? Do we need to spend time on this
in the meantime?

This selftests helps generically with that *and* helps peg on to it any sanity
checks you may wish to add to those APIs which we just don't want to do
upstream.

From my POV, it would be much more important to check if the API
works as expected. I mean that it gives the right results.

I am not sure that performance is that important to spend more time
on this one.

Also I am not sure if selftests are the right location for performance
tests. My understanding is that it is a framework for functional
testing. It is showing if the tests passed or not. But performance
tests do not give "pass or not" results.

Best Regards,
Petr