On Wed, 2016-09-14 at 12:36 +0530, Parav Pandit wrote:

Hi Dennis,

Do you know how would HFI1 driver would work along with rdma cgroup?

Keep in mind HFI1 driver has two "modes" of operation. We support
verbs, and would surely fall in line with whatever cgroups do for IB
core. For our psm interface, not sure how cgroups would come into play.
Psm is designed to expose the hw to user and avoid the kernel when
possible adding more kernel control is sort of contrary to that.

Now that being said, Christoph recently made mention of maybe having a
drivers/psm [1]. I really haven't had a chance to think about the
implications of that, but maybe it's worth considering, after all we
have two implementations, qib and hfi1. So anyway I'm not sure we need
to be too concerned about cgroups right now as far as psm side of
things goes.

Depending how things shake out for the uAPI rewrite, or verbs 2.0 or
whatever we are calling it today things may change.

[1] http://marc.info/?l=linux-rdma&m=147401714313831&w=2

-Denny

Hi Matan, Leon, Jason,
Apart from HFI1, is there any other concern?
Or Patch is good to go?

4.8 dates are close by (2 weeks) and there are two git trees involved
(that might cause merge error to Linus) so if there are no issues, I
would like to make request to Doug to consider it for 4.8 early on.

Parav

On Mon, Sep 12, 2016 at 10:37 AM, Leon Romanovsky [off-list ref]
wrote:

quoted

On Sun, Sep 11, 2016 at 11:52:35AM -0600, Jason Gunthorpe wrote:

quoted

On Sun, Sep 11, 2016 at 07:24:45PM +0200, Christoph Hellwig
wrote:

quoted

quoted

quoted

quoted

I've posted some initial work toward a) a while ago, and
once we

Did it get merged? Do you have a pointer?

http://www.spinics.net/lists/linux-rdma/msg31958.html

Right, I remember that. Certainly the right direction

quoted

quoted

However, everything under verbs is not straightforward. The
files in
userspace are not copies...

user:

struct ibv_query_device {
       __u32 command;
       __u16 in_words;
       __u16 out_words;
       __u64 response;
       __u64 driver_data[0];
};

kernel:

struct ib_uverbs_query_device {
        __u64 response;
        __u64 driver_data[0];
};

We'll obviously need different strutures for the libibvers API
and the kernel interface in this case, and we'll need to figure
out
how to properly translate them.  I think a cast, plus compile
time
type checking ala BUILD_BUG_ON is the way to go.

I'm not sure I follow, which would I cast?

BUILD_BUG_ON(sizeof(ibv_query_device) ==
sizeof(ib_uverbs_cmd_hdr) +
             sizeof(ib_uverbs_query_device))

?

quoted

quoted

I'm thinking the best way forward might be to use a script
and
transform userspace into:

struct ibv_query_device {
  struct ib_uverbs_cmd_hdr hdr;
  struct ib_uverbs_query_device cmd;
};

That would break the users of the interface.

Sorry, I mean doing this inside rdma-plumbing. Since the change
is ABI
identical the modified libibverbs would still be binary
compatible
with all providers but not source compatible. Since all kernel
supported providers are in rdma-plumbing we can add the '.cmd.'
at the
same time.

The kernel uapi header would stay the same.

quoted

However automatically generating the user ABI from the kernel
one
might still be a good idea in the long run.

My preference would be to try and use the kernel headers
directly.

I thought the same, especially after realizing that they are almost
copy/paste from the vendor *-abi.h files.

quoted

Jason

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" 
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html