RE: [ofa-general] Re: [ewg] Re: [PATCH] IB/ehca: Serialize HCA-related hCalls on POWER5

-----Original Message-----
From: Joachim Fenkes [mailto:FENKES@de.ibm.com]
Sent: Thursday, December 13, 2007 1:00 PM
To: Caitlin Bestler
Cc: Arnd Bergmann; caitlin.bestler@gmail.com; OF-General; LKML;
linuxppc-dev@ozlabs.org; Or Gerlitz; Roland Dreier; Stefan Roscher
Subject: Re: [ofa-general] Re: [ewg] Re: [PATCH] IB/ehca: Serialize
HCA-related hCalls on POWER5
=20
caitlin.bestler@gmail.com wrote on 13.12.2007 20:22:49:
=20

quoted

On Dec 13, 2007 12:30 AM, Or Gerlitz [off-list ref] wrote:

quoted

The current implementation of the open iscsi initiator makes sure

to

quoted

quoted

issue commands in thread (sleepable) context, see iscsi_xmitworker

and

quoted

quoted

references to it in drivers/scsi/libiscsi.c , so this keeps ehca

users

quoted

quoted

safe for the time being.

=20

quoted

I agree, *some* form of FMR support is important for iSER (and

probably

quoted

for NFS over RDMA as well). Rather than adding a crippled NO FMR
mode it would make more sense to add support for FMR Work Requests.
I'm not certain what, if any, impact that would have on the Power5

problem,

quoted

but that's certainly a cleaner path for iWARP.

=20
Well, FMR WRs wouldn't change the eHCA issue -- the driver would have
to
make an hCall in any case, and the architecture says that the hCalls
used
in this scenario might return H_LONG_BUSY, causing the driver to

sleep.

No
way around that. Because of this, eHCA's FMRs are actually standard

MRs

with a different API.
=20
If, as Or said, the iSCSI initiator issues commands in sleepable
context
anyway, nothing would be lost by using standard MRs as a fallback
solution
if FMRs aren't available, would it?
=20

To clarify, an FMR Work Request is simply posted to the SendQ like
any other Work Request (of course the QP has to be privileged, or
it will complete in error). An SQ Post should never block.

But yes, if the current iSCSI initiator always does all call-based
FMRs in a sleepable context then I would agree then any changes can
wait for the first vendor that wants to support FMR Work Requests.

FMR Work Requests can be pipelined, so anyone with hardware that
supported them would have strong motivation to enable the open
iSCSI initiator to take advantage of this.