On Tue, Jul 25, 2023 at 05:59:22PM +0200, Christian Brauner wrote:

On Tue, Jul 25, 2023 at 11:54:26PM +0800, Philip Li wrote:

quoted

On Tue, Jul 25, 2023 at 03:12:22PM +0000, Chuck Lever III wrote:

quoted

quoted

On Jul 22, 2023, at 4:33 PM, Chuck Lever III [off-list ref] wrote:

quoted

On Jul 17, 2023, at 2:46 AM, kernel test robot [off-list ref] wrote:


hi, Chuck Lever,

we reported a 3.0% improvement of stress-ng.handle.ops_per_sec for this commit
on
https://lore.kernel.org/oe-lkp/202307132153.a52cdb2d-oliver.sang@intel.com/ (local)

but now we noticed a regression, detail as below, FYI

Hello,

kernel test robot noticed a -15.5% regression of will-it-scale.per_thread_ops on:


commit: a1a690e009744e4526526b2f838beec5ef9233cc ("[PATCH v7 3/3] shmem: stable directory offsets")
url: https://github.com/intel-lab-lkp/linux/commits/Chuck-Lever/libfs-Add-directory-operations-for-stable-offsets/20230701-014925
base: https://git.kernel.org/cgit/linux/kernel/git/akpm/mm.git mm-everything
patch link: https://lore.kernel.org/all/168814734331.530310.3911190551060453102.stgit@manet.1015granger.net/ (local)
patch subject: [PATCH v7 3/3] shmem: stable directory offsets

testcase: will-it-scale
test machine: 104 threads 2 sockets (Skylake) with 192G memory
parameters:

nr_task: 16
mode: thread
test: unlink2
cpufreq_governor: performance


In addition to that, the commit also has significant impact on the following tests:

+------------------+-------------------------------------------------------------------------------------------------+
| testcase: change | will-it-scale: will-it-scale.per_thread_ops -40.0% regression                                   |
| test machine     | 36 threads 1 sockets Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz (Cascade Lake) with 128G memory |
| test parameters  | cpufreq_governor=performance                                                                    |
|                  | mode=thread                                                                                     |
|                  | nr_task=16                                                                                      |
|                  | test=unlink2                                                                                    |
+------------------+-------------------------------------------------------------------------------------------------+
| testcase: change | stress-ng: stress-ng.handle.ops_per_sec 3.0% improvement                                        |
| test machine     | 36 threads 1 sockets Intel(R) Core(TM) i9-9980XE CPU @ 3.00GHz (Skylake) with 32G memory        |
| test parameters  | class=filesystem                                                                                |
|                  | cpufreq_governor=performance                                                                    |
|                  | disk=1SSD                                                                                       |
|                  | fs=ext4                                                                                         |
|                  | nr_threads=10%                                                                                  |
|                  | test=handle                                                                                     |
|                  | testtime=60s                                                                                    |
+------------------+-------------------------------------------------------------------------------------------------+


If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot [off-list ref]
| Closes: https://lore.kernel.org/oe-lkp/202307171436.29248fcf-oliver.sang@intel.com (local)


Details are as below:
-------------------------------------------------------------------------------------------------->


To reproduce:

      git clone https://github.com/intel/lkp-tests.git
      cd lkp-tests
      sudo bin/lkp install job.yaml           # job file is attached in this email

Has anyone from the lkp or ltp teams had a chance to look at this?
I'm stuck without this reproducer.

Sorry about this that fedora is not fully supported now [1]. A possible way
is to run the test inside docker [2]. But we haven't fully tested the
reproduce steps in docker yet, which is in our TODO list. Also a concern is
that docker environment probably can't reproduce the performance regression.

For now, not sure whether it is convenient for you to have a ubuntu or debian
environment to give a try? Another alternative is if you have new patch, we
can assist to verify it on our machines.

So while we have your attention here. I've asked this a while ago in
another mail: It would be really really helpful if there was a way for
us to ask/trigger a perf test run for specific branches/patches we
suspect of being performance sensitive.

It's a bit of a shame that we have no simple way of submitting a custom
job and get performance results reported. I know that resources for this
are probably scarce but some way to at least request it would be really
really nice.

Apologize for this limitation. We have some mid-term TODO list to allow
the verification of reported issue (start from build report) for fix patch.

We will consider runtime side as well per this input to provide a better
experience. And we can start with a controlled scope like to queue the
test in the report, so test suite/parameters/platform is in a manageable
manner.

Thanks for the input.