Hi All,

Trying to follow the specific conversation.  It is correct, the lab does
not list the specific throughput values achieved by the hardware, as that
data can be sensitive to the hardware vendors, etc. The purpose of the lab
is to check for degradations caused by patches, so the difference is really
the important factor.  The comparison is against a prior run on the same
hardware, via the DPDK main branch, so any delta should be caused by the
specific patch changes (excluding statistical "wiggle").

If the group would prefer, we could calculate additional references if
desired (i.e. difference from the last official release, or a monthly run
of the current, etc.).  We just need the community to define their needs,
and we can add this to the development queue.

Cheers,
Lincoln


On Thu, Jan 21, 2021 at 4:29 AM Morten Brørup [off-list ref]
wrote:

quoted

From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ferruh Yigit
Sent: Thursday, January 21, 2021 10:19 AM

On 1/15/2021 6:39 PM, Ali Alnubani wrote:

quoted

Hi,
Adding Ferruh and Zhaoyan,

quoted

Ali,

You reported some performance regression, did you confirm it?
If I get no reply by monday, I'll proceed with this patch.

Sure I'll confirm by Monday.

Doesn't the regression also reproduce on the Lab's Intel servers?
Even though the check iol-intel-Performance isn't failing, I can see

that the throughput differences from expected for this patch are less
than those of another patch that was tested only 20 minutes earlier.
Both patches were applied to the same tree:

quoted

https://mails.dpdk.org/archives/test-report/2021-January/173927.html

quoted

| 64         | 512     | 1.571                               |

https://mails.dpdk.org/archives/test-report/2021-January/173919.html

quoted

| 64         | 512     | 2.698                               |

Assuming that pw86457 doesn't have an effect on this test, it looks

to me that this patch caused a regression in Intel hardware as well.

quoted

Can someone update the baseline's expected values for the Intel NICs

and rerun the test on this patch?

quoted

Zhaoyan said that the baseline is calculated dynamically,
what I understand is baseline set based on previous days performance
result, so
it shouldn't require updating.

That sounds smart!

Perhaps another reference baseline could be added, for informational
purposes only:
Deviation from the performance of the last official release.

quoted

But cc'ed the lab for more details.

-- 
*Lincoln Lavoie*
Senior Engineer, Broadband Technologies
21 Madbury Rd., Ste. 100, Durham, NH 03824
lylavoie@iol.unh.edu
https://www.iol.unh.edu
+1-603-674-2755 (m)
<https://www.iol.unh.edu>