Re: [PATCH v1] x86/lib: Optimize 8x loop and memory clobbers in csum_partial.c

On Thu, Dec 2, 2021 at 9:01 AM Eric Dumazet [off-list ref] wrote:

On Thu, Dec 2, 2021 at 6:24 AM David Laight [off-list ref] wrote:

quoted

I've dug out my test program and measured the performance of
various copied of the inner loop - usually 64 bytes/iteration.
Code is below.

It uses the hardware performance counter to get the number of
clocks the inner loop takes.
This is reasonable stable once the branch predictor has settled down.
So the different in clocks between a 64 byte buffer and a 128 byte
buffer is the number of clocks for 64 bytes.

Intuitively 10 passes is a bit low. Also you might consider aligning
the `csum64` function and possibly the loops.

There a reason you put ` jrcxz` at the beginning of the loops instead
of the end?

quoted

(Unlike the TSC the pmc count doesn't depend on the cpu frequency.)

What is interesting is that even some of the trivial loops appear
to be doing 16 bytes per clock for short buffers - which is impossible.
Checksum 1k bytes and you get an entirely different answer.
The only loop that really exceeds 8 bytes/clock for long buffers
is the adxc/adoc one.

What is almost certainly happening is that all the memory reads and
the dependant add/adc instructions are all queued up in the 'out of
order' execution unit.
Since 'rdpmc' isn't a serialising instruction they can still be
outstanding when the function returns.
Uncomment the 'rdtsc' and you get much slower values for short buffers.

Maybe add an `lfence` before / after `csum64`

quoted

When testing the full checksum function the queued up memory
reads and adc are probably running in parallel with the logic
that is handling lengths that aren't multiples of 64.

I also found nothing consistently different for misaligned reads.

These were all tested on my i7-7700 cpu.

I usually do not bother timing each call.
I instead time a loop of 1,000,000,000 calls.
Yes, this includes loop cost, but this is the same cost for all variants.
   for (i = 0; i < 100*1000*1000; i++) {
        res += csum_partial((void *)frame + 14 + 64*0, 40, 0);
        res += csum_partial((void *)frame + 14 + 64*1, 40, 0);
        res += csum_partial((void *)frame + 14 + 64*2, 40, 0);
        res += csum_partial((void *)frame + 14 + 64*3, 40, 0);
        res += csum_partial((void *)frame + 14 + 64*4, 40, 0);
        res += csum_partial((void *)frame + 14 + 64*5, 40, 0);
        res += csum_partial((void *)frame + 14 + 64*6, 40, 0);
        res += csum_partial((void *)frame + 14 + 64*7, 40, 0);
        res += csum_partial((void *)frame + 14 + 64*8, 40, 0);
        res += csum_partial((void *)frame + 14 + 64*9, 40, 0);
    }

+ 1. You can also feed `res` from previous iteration to the next
iteration to measure latency cheaply if that is better
predictor of performance.

Then use " perf stat ./bench"   or similar.