From: David Miller <davem@davemloft.net> Date: 2014-08-25 23:12:26
During the Networking Workshop I mentioned converting the inet hash
tables over to rhashtable so that we don't allocate this insanely
large hash table at boot time which goes largely unused.
I took a quick look at this last night and the only thing we really
need is the addition of a set of rhashtable interfaces which use
NULLs lists, as the inet hashtables currently require.
Also, I noticed in the netlink changes this really expensive
synchronize_net() added to netlink_release(), is that _really_
necessary?
That's really expensive and my impression was that such a sync is only
needed during hash table resizing, not when getting rid of objects
that we in an rhashtable.
Thomas?
From: Thomas Graf <tgraf@suug.ch> Date: 2014-08-25 23:33:47
On 08/25/14 at 04:12pm, David Miller wrote:
During the Networking Workshop I mentioned converting the inet hash
tables over to rhashtable so that we don't allocate this insanely
large hash table at boot time which goes largely unused.
I took a quick look at this last night and the only thing we really
need is the addition of a set of rhashtable interfaces which use
NULLs lists, as the inet hashtables currently require.
Eric brought this up as well last week. I see no problem using
a non-NULL token as the default to identify the end of the list.
We had a quick sitdown with Paul E. McKenney and came up with
some ideas that should allow doing that even with resizes taking
place by using the hash of the entry as the token.
We also discussed the possibility to do the resizing outside of
the insert/remove context and move it to a worker thread using
per bucket locks. We think that we may have found something that
might work and I will give that a shot. It will allow to use a
resizing rhashtable with the insert/remove being in atomic
context which I believe is needed for the inet cache.
Also, I noticed in the netlink changes this really expensive
synchronize_net() added to netlink_release(), is that _really_
necessary?
That's really expensive and my impression was that such a sync is only
needed during hash table resizing, not when getting rid of objects
that we in an rhashtable.
The reason I added added the sync is because I could not see how
else to prevent the sock_put() in netlink_release() to release
socket memoray, specifically the embedded rhash_head, that is
possibly being accessed in a RCU protected reader traversing the
bucket. Such a reader would not hold a reference to the socket.
I may be missing something though and I'm happy to change this for
something better.
From: Eric Dumazet <hidden> Date: 2014-08-25 23:35:37
On Mon, 2014-08-25 at 16:12 -0700, David Miller wrote:
During the Networking Workshop I mentioned converting the inet hash
tables over to rhashtable so that we don't allocate this insanely
large hash table at boot time which goes largely unused.
I took a quick look at this last night and the only thing we really
need is the addition of a set of rhashtable interfaces which use
NULLs lists, as the inet hashtables currently require.
Also, I noticed in the netlink changes this really expensive
synchronize_net() added to netlink_release(), is that _really_
necessary?
That's really expensive and my impression was that such a sync is only
needed during hash table resizing, not when getting rid of objects
that we in an rhashtable.
Thomas?
After the workshop we had a discussion with Paul McKenney, Thomas, and
John where we discussed all this.
An other issue we raised was that the grow should happen in process
context, while inserts and deletes should happen from softirq.
We also mentioned the need of a per bucket spinlock, and keep the mutex
only to protect the resizes. (Or an array of spinlocks as used by TCP)
Paul gave a lot of ideas and this seems feasible.
Thomas agreed to work on all this. If not I can do this myself.
(Note that conntrack also uses the same nulls rcu hash tables)
From: David Miller <davem@davemloft.net> Date: 2014-08-25 23:36:46
From: Thomas Graf <tgraf@suug.ch>
Date: Tue, 26 Aug 2014 00:33:46 +0100
The reason I added added the sync is because I could not see how
else to prevent the sock_put() in netlink_release() to release
socket memoray, specifically the embedded rhash_head, that is
possibly being accessed in a RCU protected reader traversing the
bucket. Such a reader would not hold a reference to the socket.
I may be missing something though and I'm happy to change this for
something better.
Ok, this is going to be a tree-wide issue where we try to use
rhashtable with sockets of any type.
Perhaps we're overdue for RCU freeing of sockets, but that added
noticable latency last time I tried it.