Hi Max,

On 2021/10/27 18:49, Max Gurtovoy wrote:

Hi Ruozhu,

The performance tests we did for all the IO sizes showed better 
performance with local invalidation in most of the cases (mainly for 
large IOs, the remote invalidation task is harder for the HW).

Maybe there is 1 or 2 cases, such as small IO latency, that improves 
with remote-invalidation but for now we decided to disable it.

I test it with single thread 4K write IO, maybe this is why I get better 
latency data.

I will test various IO models when I have the opportunity, and then give 
you feedback.

In addition, I would like to know if the remote invalidation task is 
harder for the HW, Why do we need to enable it in the general(no t10-pi) 
scene?

Thanks,
Ruozhu

I guess we can add a capability for the device that will indicate it 
in the future.

What is the difference you see ? and what is the scenario ?

Thanks,

-Max.

On 10/27/2021 11:50 AM, liruozhu wrote:

quoted

Hi Max,

I was testing T10-PI feature of nvme-rdma recently, and found that 
when using T10-PI, nvme-rdma driver did not set 
NVME_SGL_FMT_INVALIDATE flag in the sgl type field, so the host 
software needs to do invalid rkey by itself. I tried to add the flag 
to test it. Turns out T10-PI feature is still working, and I get 
better IO latency with it enable.
I read the original patches in the mailing list, and found that patch 
v1 set this flag. But it was silent dropped on patch v5 without any 
comments. Is it for any special considerations to delete it?

Thanks,
Ruozhu

.