Hi,

Sorry for the delay, I have been OOO for the past few days.

Indeed the underlying transport is Ethernet, and we have found that flow 
control is disabled on the switch side.
I still do not have access to the system for re-tests with the flow 
control, and 4.10. But I am working on assembling another system.

A little off-topic, but something that may help me get the other system 
up faster, can I use a dual CX4 card as initiator and target (i.e. one 
port going up to the switch, and the other coming back into the system) 
without the kernel looping back? I have a spare CX4, so if possible, I 
will use it to build a mini system for recreates of this sort.

Thanks,
Shahar


On 03/07/2017 03:41 PM, Sagi Grimberg wrote:

quoted

Hi,

Shahar/Joseph, what is your link layer conf (IB/Eth) ?
In eth case, have you configured some PFC ? if not, can you try it ?
I suspect that this is the root cause

The root cause is that the device fails frwr in retransmission
flow, if PFC is not on, it will happen almost immediately, if not
it will happen at some point...

quoted

and it might help you avoiding
this case, meanwhile we're looking for for the best solution.

Adding Vladimir that will run iSER on his performance setup with the new
fencing patch (not an NVMEoF related issue).
We can run also NVMEoF later on if needed.

Thanks.