From: Martin Sustrik <hidden> Date: 2013-02-08 12:43:04
On 07/02/13 23:44, Andrew Morton wrote:
That's a nice changelog but it omitted a critical thing: why do you
think the kernel needs this feature? What's the value and use case for
being able to poll these descriptors?
To address the question, I've written down detailed description of the
challenges of the network protocol development in user space and how the
proposed feature addresses the problems.
It's too long to fit into ChangeLog, but it may be worth reading when
trying to judge the merit of the patch.
It can be found here: http://www.250bpm.com/blog:16
Martin
From: Eric Wong <hidden> Date: 2013-02-08 22:21:08
Martin Sustrik [off-list ref] wrote:
On 07/02/13 23:44, Andrew Morton wrote:
quoted
That's a nice changelog but it omitted a critical thing: why do you
think the kernel needs this feature? What's the value and use case for
being able to poll these descriptors?
To address the question, I've written down detailed description of
the challenges of the network protocol development in user space and
how the proposed feature addresses the problems.
It's too long to fit into ChangeLog, but it may be worth reading
when trying to judge the merit of the patch.
It can be found here: http://www.250bpm.com/blog:16
Using one eventfd per userspace socket still seems a bit wasteful.
Couldn't you use a single pipe for all sockets and write the efd_mask to
the pipe for each socket?
A read from the pipe would behave like epoll_wait.
You might need to use one-shot semantics; but that's probably
the easiest thing in multithreaded apps anyways.
From: Martin Sustrik <hidden> Date: 2013-02-09 02:40:35
Hi Eric,
On 08/02/13 23:21, Eric Wong wrote:
Martin Sustrik[off-list ref] wrote:
quoted
On 07/02/13 23:44, Andrew Morton wrote:
quoted
That's a nice changelog but it omitted a critical thing: why do you
think the kernel needs this feature? What's the value and use case for
being able to poll these descriptors?
To address the question, I've written down detailed description of
the challenges of the network protocol development in user space and
how the proposed feature addresses the problems.
It's too long to fit into ChangeLog, but it may be worth reading
when trying to judge the merit of the patch.
It can be found here: http://www.250bpm.com/blog:16
Using one eventfd per userspace socket still seems a bit wasteful.
Wasteful in what sense? Occupying a slot in file descriptor table?
That's the price for having the socket uniquely identified by the fd.
Couldn't you use a single pipe for all sockets and write the efd_mask to
the pipe for each socket?
A read from the pipe would behave like epoll_wait.
You might need to use one-shot semantics; but that's probably
the easiest thing in multithreaded apps anyways.
Having multiple sockets represented by a single eventfd. how would you
distinguish where did individual events came from?
struct pollfd pfd;
...
poll (pfd, 1, -1);
if (pfd.revents & POLLIN) /* Incoming data on which socket? */
...
Martin
From: Eric Wong <hidden> Date: 2013-02-09 03:54:31
Martin Sustrik [off-list ref] wrote:
On 08/02/13 23:21, Eric Wong wrote:
quoted
Martin Sustrik[off-list ref] wrote:
quoted
To address the question, I've written down detailed description of
the challenges of the network protocol development in user space and
how the proposed feature addresses the problems.
It can be found here: http://www.250bpm.com/blog:16
Using one eventfd per userspace socket still seems a bit wasteful.
Wasteful in what sense? Occupying a slot in file descriptor table?
That's the price for having the socket uniquely identified by the
fd.
Yes. I realize eventfd is small, but I don't think eventfd is needed
at all, here. Just one pipe.
quoted
Couldn't you use a single pipe for all sockets and write the efd_mask to
the pipe for each socket?
A read from the pipe would behave like epoll_wait.
You might need to use one-shot semantics; but that's probably
the easiest thing in multithreaded apps anyways.
Having multiple sockets represented by a single eventfd. how would
you distinguish where did individual events came from?
struct pollfd pfd;
...
poll (pfd, 1, -1);
if (pfd.revents & POLLIN) /* Incoming data on which socket? */
...
No eventfd, you write just write struct to the pipe, and consume the
struct to a fixed size buffer:
/* trigger readiness notification for sock,
* this probably needs a lock around it
*/
void sock_trigger(struct my_sock *sock, int events)
{
struct efd_mask mask;
/* check if the triggeered event is something sock wants: */
events &= sock->watched_events;
if (!events)
return;
mask.events = events;
mask.ptr = sock;
/*
* preventing sock from being in the pipe multiple times
* is probably required (or just a good idea). Which is
* why I mentioned oneshot semantics are probably required.
*/
if (oneshot)
sock->watched_events = 0;
/*
* This is analogous to:
* list_add_tail(&epi->rdllink, &ep->rdllist);
* in fs/eventpoll.c
*
* This may block, but that's why consumer_loop runs in different
* threads. Or run some iteration of consumer_loop here if
* it blocks (beware of stack depth from recursion, though)
*/
write(pipe_wr, &mask, sizeof(mask));
}
/* in another thread (or several threads) */
void consumer_loop(int pipe_rd)
{
struct efd_mask mask;
struct my_sock *sock;
for (;;) {
/*
* analogous to:
* epoll_wait(.., maxevents=1, ...);
*
* You can read several masks at once if have one thread,
* but I usually use maxevents=1 (+several threads) to
* distribute traffic between threads
*/
read(pipe_rd, &mask, sizeof(mask));
sock = mask.ptr;
if (mask.events & POLLIN)
sock_read(sock);
else if (mask.events & POLLOUT)
sock_write(sock);
...
/* analogous to epoll_ctl() */
if (sock->write_buffered)
sock->watched_events |= POLLOUT;
if (sock->wants_more_data)
sock->watched_events |= POLLIN;
/* onto the next ready event */
}
}
From: Martin Sustrik <hidden> Date: 2013-02-09 07:36:10
On 09/02/13 04:54, Eric Wong wrote:
quoted
quoted
Using one eventfd per userspace socket still seems a bit wasteful.
Wasteful in what sense? Occupying a slot in file descriptor table?
That's the price for having the socket uniquely identified by the
fd.
Yes. I realize eventfd is small, but I don't think eventfd is needed
at all, here. Just one pipe.
Ah. Got you! You mean not to change the kernel, just use pipe for the
purpose.
However, the convoluted pipe-style design is the problem I am trying to
solve rather than the solution. It leads to convoluted APIs with
convoluted semantics as described in the article. I've been using that
kind of design for past 8 years and every time I have to deal with it I
swear that one day I will implement a proper in-kernel solution to get
rid of the hack.
And now I have finally done so.
Martin
From: Eric Wong <hidden> Date: 2013-02-09 11:51:50
Martin Sustrik [off-list ref] wrote:
On 09/02/13 04:54, Eric Wong wrote:
quoted
quoted
quoted
Using one eventfd per userspace socket still seems a bit wasteful.
Wasteful in what sense? Occupying a slot in file descriptor table?
That's the price for having the socket uniquely identified by the
fd.
Yes. I realize eventfd is small, but I don't think eventfd is needed
at all, here. Just one pipe.
Ah. Got you! You mean not to change the kernel, just use pipe for
the purpose.
However, the convoluted pipe-style design is the problem I am trying
to solve rather than the solution. It leads to convoluted APIs with
convoluted semantics as described in the article. I've been using
that kind of design for past 8 years and every time I have to deal
with it I swear that one day I will implement a proper in-kernel
solution to get rid of the hack.
And now I have finally done so.
Yes, your eventfd change is probably the best way if you want/need
to only watch a subset of your sockets, especially if you want
poll/select to be an option.
From: Martin Sustrik <hidden> Date: 2013-02-09 12:04:27
On 2013-02-09 12:51, Eric Wong wrote:
Yes, your eventfd change is probably the best way if you want/need
to only watch a subset of your sockets, especially if you want
poll/select to be an option.
Yes, the poll/select thing is the important point.
I wouldn't care if the only problem was that I, as the protocol
implementer, would have to implement some kind of workaround in my
protocol library. The problem is that these convoluted semantics leak --
through the use of poll, select et al. -- to the end user.
From my personal experience I can say that end users have pretty hard
time using such complex workarounds instead of simply using a native
file descriptor with standardised semantics.
Martin