[RFC|PATCH] AF_UNIX multicast capability for 2.5.66

For those who didn't read the first couple threads, this patch adds multicast functionality to unix sockets, in similar fashion (and using a similar API) as UDP multicast, but easier to use.

To use the functionality, in userspace you would allocate and bind a socket as normal in the AF_UNIX family, and then you would use setsockop() to associate yourself with one or more multicast addresses, also in the AF_UNIX family.  Any message sent to a multicast address gets duplicated by the kernel and distributed to all processes associated with that address.

If an address already exists and is not multicast, you cannot associate yourself with it using setsockopt(), and if it exists and is multicast, you cannot bind() do it.  All AF_UNIX addresses exist in the same namespace and must be either multicast or unicast.

It does not make sense to allow streaming to a multicast address, so I plan on disallowing this (and other similar things) in a future release of the patch. I'm not sure about allowing datagram sockets to connect() to a multicast address, I haven't looked at the code in depth.

At any rate, here are the results of some testing on the latest kernel comparing kernel multicast against a userspace solution.  There is a sender program which sends messages of various sizes to various numbers of listeners.  Each message has a timestamp embedded within it, and the listeners determine the latency between the sending and receiving of the message.  In the userspace solution, the list of listeners is kept in a shared memory area, which is faster than using a distributor process.

In the interests of figuring out the best possible performance I've changed the testing methodology from the last bunch of tests, and the organization of the results is slightly different to make it easier to add new results. In this test sequence the sender and receiver processes have been run with nice -20 to minimize interference from other entities in the system.  These are best-case results from a number of runs, but the numbers are fairly consistant across runs.


data size and kernel                 number of listeners
44 bytes                   10       20          50          100        200
2.5.66 userspace        134,297    206,561   416,1401     761,2824    1500,5711
2.5.66 kernelspace      75,232     119,457   213,1142     356,2308    679,4710

236 bytes
2.5.66 userspace       143,306    218,584    447,1472     814,3013    1399,6007
2.5.66 kernelspace      80,244    115,469    216,1176     365,2371    682,4893

40036 bytes
2.5.66 userspace        478,3613  497,7405  496,18114    487,36759    518,74566
2.5.66 kernelspace      287,1672  327,3299  444,8129     663,16125   1000,31937


The numbers definately favour a kernel-space solution.  That said, it would be possible to implement this using UDP messaging, but UDP latency is generally about 30 percent higher than AF_UNIX on my 1.8GHz P4, and it's more of a hassle to configure UDP multicast.

I would appreciate any comments on the patch, if you see any technical bugs or if you think there is a better way to do something please do let me know.  I'm sure there's some fine point about locking that I missed, or something similar.

Thanks,

Chris




-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com