Re: ibv_fork_init() question
From: Roland Dreier <hidden>
Date: 2012-02-27 19:58:45
On Mon, Feb 27, 2012 at 11:54 AM, Steve Wise [off-list ref] wrote:
I'm experimenting with doing RDMA operations from a child process and I have a question. My application calls ibv_fork_init() at startup (first thing in main()) and it succeeds, and I see, for instance, that libibverbs has initialized mm_root. Next my application registers page-aligned memory that is 8KB in size. The memory was allocated with memalign(). Then the app sets up a connection to an RDMA peer. Next the application calls fork(). The parent does a wait() and the child does an rdma read operation into this registered memory from a valid mr of the peer. The read succeeds from the rdma device perspective. But when the child process tries to read the memory (to verify the rdma operation worked), I get a seg fault. Q: Am I doing something incorrect here? I can't find much documentation on how this all should work.
Yeah, unfortunately this isn't really intended to work and I don't have a good suggestion on how to make it work. Even with the libibverbs fork support, the expectation is that all RDMA stuff will be done from the master process. All that the ibv_fork_init() call does is trigger tracking so that the address space that is used for RDMA isn't mapped into the child process at all. So yeah, what you see happening is exactly what I would expect. - R. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html