Re: Understanding mutual exclusion between rtnl_lock and rcu_read_lock

On Feb 2, 2017, at 5:52 PM, Alexander Duyck [off-list ref] wrote:

On Thu, Feb 2, 2017 at 3:47 PM, Joel Cunningham [off-list ref] wrote:

quoted

Hi,

I’m studying the synchronization used on different parts of struct net_device and I’m struggling to understand how structure member modifications in dev_ioctl are synchronized.  Getters in dev_ifsioc_locked() are only holding rcu_read_lock() while setters in dev_ifsioc() are holding rtnl_lock, but not using RCU APIs.  I was specifically looking at SIOCGIFHWADDR/SIOCSIFHWADDR.  What’s to prevent one CPU from executing a getter and another CPU from executing a setter resulting in possibly a torn read/write?  I didn’t see anything in rtnl_lock() that would wait for any rcu_reader_lock() critical sections (on other CPUs) to finish before acquiring the mutex.

Is there something about dev_ioctl that prevents parallel execution? or maybe something I still don’t understand about the RCU implementation?

Thanks,

Joel

My advice would be to spend more time familiarizing yourself with RCU.
The advantage of RCU is that it allows for updates while other threads
are accessing the data.

Thanks for the follow up!

I have been trying to find more examples of RCU where the writer updates the structure in-place without using RCU APIs, but so far haven’t found anything

In the case of SIOCSIFHWADDR, we get a pointer to the net_device through __dev_get_by_name() and then pass it to dev_set_mac_address() to modify through ndo_set_mac_address().  I didn’t see any uses of RCU APIs on the writer side and that’s why I figured there was something going on with rtnl_lock() that I didn’t understand or that the dev_ioctl function wasn’t re-entrant from another CPU

The rtnl_lock is just meant to prevent
multiple writers from updating the data simultaneously.  So between
writers the rtnl_lock is used to keep things synchronized, but between
writers and readers the mechanism that is meant to protect the data
and keep it sane is RCU.

Your description of rtnl_lock make sense.  I’m still confused with how the setter side code (in dev_ifsioc) is using RCU since I don’t see any APIs called.

Thanks,

Joel