Re: [PATCH 0/3] epoll: Add epoll_pwait1 syscall
From: Andy Lutomirski <hidden>
Date: 2015-01-09 01:28:45
On Thu, Jan 8, 2015 at 5:25 PM, Fam Zheng [off-list ref] wrote:
On Thu, 01/08 09:57, Andy Lutomirski wrote:quoted
On Thu, Jan 8, 2015 at 1:12 AM, Miklos Szeredi [off-list ref] wrote:quoted
On Thu, 2015-01-08 at 16:25 +0800, Fam Zheng wrote:quoted
Applications could use epoll interface when then need to poll a big number of files in their main loops, to achieve better performance than ppoll(2). Except for one concern: epoll only takes timeout parameters in microseconds, rather than nanoseconds. That is a drawback we should address. For a real case in QEMU, we run into a scalability issue with ppoll(2) when many devices are attached to guest, in which case many host fds, such as virtual disk images and sockets, need to be polled by the main loop. As a result we are looking at switching to epoll, but the coarse timeout precision is a trouble, as explained below. We're already using prctl(PR_SET_TIMERSLACK, 1) which is necessary to implement timers in the main loop; and we call ppoll(2) with the next firing timer as timeout, so when ppoll(2) returns, we know that we have more work to do (either handling IO events, or fire a timer callback). This is natual and efficient, except that ppoll(2) itself is slow. Now that we want to switch to epoll, to speed up the polling. However the timer slack setting will be effectively undone, because that way we will have to round up the timeout to microseconds honoring timer contract. But consequently, this hurts the general responsiveness. Note: there are two alternatives, without changing kernel: 1) Leading ppoll(2), with the epollfd only and a nanosecond timeout. It won't be slow as one fd is polled. No more scalability issue. And if there are events, we know from ppoll(2)'s return, then we do the epoll_wait(2) with timeout=0; otherwise, there can't be events for the epoll, skip the following epoll_wait and just continue with other work. 2) Setup and add a timerfd to epoll, then we do epoll_wait(..., timeout=-1). The timerfd will hopefully force epoll_wait to return when it timeouts, even if no other events have arrived. This will inheritly give us timerfd's precision. Note that for each poll, the desired timeout is different because the next timer is different, so that, before each epoll_wait(2), there will be a timerfd_settime syscall to set it to a proper value. Unfortunately, both approaches require one more syscall per iteration, compared to the original single ppoll(2), cost of which is unneglectable when we talk about nanosecond granularity.I'd like to see a more ambitious change, since the timer isn't the only problem like this. Specifically, I'd like a syscall that does a list of epoll-related things and then waits. The list of things could include, at least: - EPOLL_CTL_MOD actions: level-triggered epoll users are likely to want to turn on and off their requests for events on a somewhat regular basis.This sounds good to me.quoted
- timerfd_settime actions: this allows a single syscall to wait and adjust *both* monotonic and real-time wakeups.I'm not sure, doesn't this break orthogonality between epoll and timerfd?
Yes. It's not very elegant, and more elegant ideas are welcome.
quoted
Would this make sense? It could look like: int epoll_mod_and_pwait(int epfd, struct epoll_event *events, int maxevents, struct epoll_command *commands, int ncommands, const sigset_t *sigmask);What about flags?
No room. Maybe it should just be a struct for everything instead of separate args.
Fam
-- Andy Lutomirski AMA Capital Management, LLC