On Sun, Jan 14, 2018 at 11:32:57AM +0000, James Chapman wrote:
quoted hunk
SIOCKCMATTACH writes a connected socket's sk_user_data for its own
use. Prevent it doing so if the socket's sk_user_data is already set
since some sockets (e.g. encapsulated sockets) use sk_user_data
internally.
Isn't that racy? What if sk_user_data was concurrently set right after
this test?
Also, it looks like we could create a UDP socket, attach it to KCM,
then create an L2TP tunnel on this same UDP socket. l2tp_tunnel_create()
or setup_udp_tunnel_sock() would unconditionally overwrite
sk_user_data, which will probably confuse KCM.
Tom, if I understand KCM correctly, it only makes sense to attach it to
SOCK_STREAM sockets. Shouldn't that be enforced? Maybe we should
restrict it even further, so that only known KCM-safe sockets could be
attached (that is, reject anything that isn't AF_INET* | SOCK_STREAM).
From: Eric Dumazet <hidden> Date: 2018-01-16 17:11:01
On Tue, 2018-01-16 at 11:44 +0100, Guillaume Nault wrote:
Tom, if I understand KCM correctly, it only makes sense to attach it to
SOCK_STREAM sockets. Shouldn't that be enforced? Maybe we should
restrict it even further, so that only known KCM-safe sockets could be
attached (that is, reject anything that isn't AF_INET* | SOCK_STREAM).
From: David Miller <davem@davemloft.net> Date: 2018-01-16 19:00:54
From: Tom Herbert <redacted>
Date: Tue, 16 Jan 2018 09:36:41 -0800
sk_user_data is set with the sk_callback lock held in code below.
Should be able to take the lock earlier can do this check under the
lock.
csock, and this csk, is obtained from an arbitrary one of the
process's FDs. It can be any socket type or family, and that socket's
family might set sk_user_data without the callback lock.
The only socket type check is making sure it is not another PF_KCM
socket. So that doesn't help with this problem.
From: James Chapman <jchapman@katalix.com> Date: 2018-01-17 11:13:38
On 16 January 2018 at 19:00, David Miller [off-list ref] wrote:
From: Tom Herbert <redacted>
Date: Tue, 16 Jan 2018 09:36:41 -0800
quoted
sk_user_data is set with the sk_callback lock held in code below.
Should be able to take the lock earlier can do this check under the
lock.
csock, and this csk, is obtained from an arbitrary one of the
process's FDs. It can be any socket type or family, and that socket's
family might set sk_user_data without the callback lock.
The only socket type check is making sure it is not another PF_KCM
socket. So that doesn't help with this problem.
Is it the intention to update all socket code over time to write
sk_user_data within the sk_callback lock? If so, I'm happy to address
that in the l2tp code (and update the kcm patch to check sk_user_data
within the sk_callback lock). Or is the preferred solution to restrict
KCM to specific socket families, as suggested by Guillaume earlier in
the thread?
From: David Miller <davem@davemloft.net> Date: 2018-01-17 19:25:41
From: James Chapman <jchapman@katalix.com>
Date: Wed, 17 Jan 2018 11:13:33 +0000
On 16 January 2018 at 19:00, David Miller [off-list ref] wrote:
quoted
From: Tom Herbert <redacted>
Date: Tue, 16 Jan 2018 09:36:41 -0800
quoted
sk_user_data is set with the sk_callback lock held in code below.
Should be able to take the lock earlier can do this check under the
lock.
csock, and this csk, is obtained from an arbitrary one of the
process's FDs. It can be any socket type or family, and that socket's
family might set sk_user_data without the callback lock.
The only socket type check is making sure it is not another PF_KCM
socket. So that doesn't help with this problem.
Is it the intention to update all socket code over time to write
sk_user_data within the sk_callback lock? If so, I'm happy to address
that in the l2tp code (and update the kcm patch to check sk_user_data
within the sk_callback lock). Or is the preferred solution to restrict
KCM to specific socket families, as suggested by Guillaume earlier in
the thread?
I think we have a more fundamental issue here.
sk->sk_user_data is a place where RPC layer specific data is hung off
of. By this definition SunRPC, RXRPC, RDS, TIPC, and KCM are all
using it correctly.
Phonet has a similar issue to the one seen here, it tests and changes
sk_user_data under lock_sock(). The only requirement it makes is
that the socket type is not SOCK_STREAM. However, this one might be OK
since only pep_sock sockets can be passed down into gprs_attach().
Most of these cases like SunRPC, RXRPC, etc. are fine because they
only graft on top of TCP and UDP sockets.
The weird situation here is that L2TP does tunneling and stores it's
private state in sk->sk_user_data like an RPC layer would. And KCM
allows basically any socket type to be attached.
The RPC layers create their sockets internally, so I cannot see a way
that those can be sent to a KCM attach operations. And I think that
is why this RPC invariant is important for sk_user_data usage.
If all else was equal, even though it doesn't make much sense to KCM
attach L2TP sockets to KCM, I would suggest to change L2TP to store
it's private stuff elsewhere.
But that is not the case. Anything using the generic UDP
encapsulation layer is going to make use of sk->sk_user_data like this
(see setup_udp_tunnel_sock).
It looks like over time we've accumulated this new class of uses
of sk->sk_user_data, ho hum...
And it's not like we can add a test to KCM to avoid these socket
types, because they will look like normal UDP datagram sockets.
What a mess...
Furthermore, even if you add a test to KCM, you will now need to
add the same test to L2TP and anything else which uses sk_user_data
for tunneling and for which userspace has access to the socket fd.
And it will be racy, indeed, until all such users align to the same
precise locking scheme for tests and updates to sk_user_data.
Again, what a mess...
On Wed, Jan 17, 2018 at 02:25:38PM -0500, David Miller wrote:
From: James Chapman <jchapman@katalix.com>
Date: Wed, 17 Jan 2018 11:13:33 +0000
quoted
On 16 January 2018 at 19:00, David Miller [off-list ref] wrote:
quoted
From: Tom Herbert <redacted>
Date: Tue, 16 Jan 2018 09:36:41 -0800
quoted
sk_user_data is set with the sk_callback lock held in code below.
Should be able to take the lock earlier can do this check under the
lock.
csock, and this csk, is obtained from an arbitrary one of the
process's FDs. It can be any socket type or family, and that socket's
family might set sk_user_data without the callback lock.
The only socket type check is making sure it is not another PF_KCM
socket. So that doesn't help with this problem.
Is it the intention to update all socket code over time to write
sk_user_data within the sk_callback lock? If so, I'm happy to address
that in the l2tp code (and update the kcm patch to check sk_user_data
within the sk_callback lock). Or is the preferred solution to restrict
KCM to specific socket families, as suggested by Guillaume earlier in
the thread?
I think we have a more fundamental issue here.
sk->sk_user_data is a place where RPC layer specific data is hung off
of. By this definition SunRPC, RXRPC, RDS, TIPC, and KCM are all
using it correctly.
Phonet has a similar issue to the one seen here, it tests and changes
sk_user_data under lock_sock(). The only requirement it makes is
that the socket type is not SOCK_STREAM. However, this one might be OK
since only pep_sock sockets can be passed down into gprs_attach().
But, if I read it correctly, that doesn't prevent it from being passed
to kcm_attach() later on, which will overwrite sk_user_data (unless we
update the locking scheme and refuse to overwrite sk_user_data in a
race-free way).
BTW couldn't the gprs_dev pointer be embedded in struct pep_sock?
This way pep_sk(sk)->gp could be used instead of sk->sk_user_data.
That'd probably be a violation of the phonet's layering, as that'd
tie gprs_dev to pep sockets. OTOH, only pep sockets can currently be
attached to gprs_dev, so in practice that might be a reasonable
compromise.
Most of these cases like SunRPC, RXRPC, etc. are fine because they
only graft on top of TCP and UDP sockets.
The weird situation here is that L2TP does tunneling and stores it's
private state in sk->sk_user_data like an RPC layer would. And KCM
allows basically any socket type to be attached.
The RPC layers create their sockets internally, so I cannot see a way
that those can be sent to a KCM attach operations. And I think that
is why this RPC invariant is important for sk_user_data usage.
SunRPC seems to possibly set sk_user_data on user sockets: svc_addsock()
gets a socket using sockfd_lookup() then passes it to svc_setup_socket()
which in turn sets sk_user_data. I don't know anything about SunRPC, so
I might very well have missed important details, but I believe such a
socket could be passed to KCM which could lead to the same kind of
issues as for L2TP. Other RPCs look safe to me.
If all else was equal, even though it doesn't make much sense to KCM
attach L2TP sockets to KCM, I would suggest to change L2TP to store
it's private stuff elsewhere.
But that is not the case. Anything using the generic UDP
encapsulation layer is going to make use of sk->sk_user_data like this
(see setup_udp_tunnel_sock).
Most UDP encapsulations only use kernel sockets though. It seems that
only L2TP and GTP use setup_udp_tunnel_sock() with userpsace sockets.
So it might be feasible to restrict usage of sk_user_data to kernel
sockets only.
For L2TP, we probably can adapt l2tp_sock_to_tunnel() so that it does
a lookup in a hashtable indexed by the socket pointer, rather than
dereferencing sk_user_data. That doesn't look very satisfying to me,
but that's the only way I found so far.
We also have another user of sk_user_data in l2tp_ppp, but since it
uses its own socket type, I guess we could simply embed the pointer in
its parent structure.
It looks like over time we've accumulated this new class of uses
of sk->sk_user_data, ho hum...
And it's not like we can add a test to KCM to avoid these socket
types, because they will look like normal UDP datagram sockets.
What a mess...
Furthermore, even if you add a test to KCM, you will now need to
add the same test to L2TP and anything else which uses sk_user_data
for tunneling and for which userspace has access to the socket fd.
And it will be racy, indeed, until all such users align to the same
precise locking scheme for tests and updates to sk_user_data.
Again, what a mess...
So, if I understand correctly, we can either restrict sk_user_data to
kernel sockets so that KCM couldn't act on them (but then why would we
make an exception for KCM and allow it to set sk_user_data on
non-kernel sockets?).
Or we could agree on a locking scheme for sk_user_data and update all
users so that they'd fail instead of overwriting it when it's not NULL.
Assuming my understanding is correct, do you have any preference for
fixing this issue? Or any other ideas?
From: James Chapman <jchapman@katalix.com> Date: 2018-01-18 15:56:33
On 18 January 2018 at 15:18, Guillaume Nault [off-list ref] wrote:
On Wed, Jan 17, 2018 at 02:25:38PM -0500, David Miller wrote:
quoted
From: James Chapman <jchapman@katalix.com>
Date: Wed, 17 Jan 2018 11:13:33 +0000
quoted
On 16 January 2018 at 19:00, David Miller [off-list ref] wrote:
quoted
From: Tom Herbert <redacted>
Date: Tue, 16 Jan 2018 09:36:41 -0800
quoted
sk_user_data is set with the sk_callback lock held in code below.
Should be able to take the lock earlier can do this check under the
lock.
csock, and this csk, is obtained from an arbitrary one of the
process's FDs. It can be any socket type or family, and that socket's
family might set sk_user_data without the callback lock.
The only socket type check is making sure it is not another PF_KCM
socket. So that doesn't help with this problem.
Is it the intention to update all socket code over time to write
sk_user_data within the sk_callback lock? If so, I'm happy to address
that in the l2tp code (and update the kcm patch to check sk_user_data
within the sk_callback lock). Or is the preferred solution to restrict
KCM to specific socket families, as suggested by Guillaume earlier in
the thread?
I think we have a more fundamental issue here.
sk->sk_user_data is a place where RPC layer specific data is hung off
of. By this definition SunRPC, RXRPC, RDS, TIPC, and KCM are all
using it correctly.
Phonet has a similar issue to the one seen here, it tests and changes
sk_user_data under lock_sock(). The only requirement it makes is
that the socket type is not SOCK_STREAM. However, this one might be OK
since only pep_sock sockets can be passed down into gprs_attach().
But, if I read it correctly, that doesn't prevent it from being passed
to kcm_attach() later on, which will overwrite sk_user_data (unless we
update the locking scheme and refuse to overwrite sk_user_data in a
race-free way).
BTW couldn't the gprs_dev pointer be embedded in struct pep_sock?
This way pep_sk(sk)->gp could be used instead of sk->sk_user_data.
That'd probably be a violation of the phonet's layering, as that'd
tie gprs_dev to pep sockets. OTOH, only pep sockets can currently be
attached to gprs_dev, so in practice that might be a reasonable
compromise.
quoted
Most of these cases like SunRPC, RXRPC, etc. are fine because they
only graft on top of TCP and UDP sockets.
The weird situation here is that L2TP does tunneling and stores it's
private state in sk->sk_user_data like an RPC layer would. And KCM
allows basically any socket type to be attached.
The RPC layers create their sockets internally, so I cannot see a way
that those can be sent to a KCM attach operations. And I think that
is why this RPC invariant is important for sk_user_data usage.
SunRPC seems to possibly set sk_user_data on user sockets: svc_addsock()
gets a socket using sockfd_lookup() then passes it to svc_setup_socket()
which in turn sets sk_user_data. I don't know anything about SunRPC, so
I might very well have missed important details, but I believe such a
socket could be passed to KCM which could lead to the same kind of
issues as for L2TP. Other RPCs look safe to me.
quoted
If all else was equal, even though it doesn't make much sense to KCM
attach L2TP sockets to KCM, I would suggest to change L2TP to store
it's private stuff elsewhere.
But that is not the case. Anything using the generic UDP
encapsulation layer is going to make use of sk->sk_user_data like this
(see setup_udp_tunnel_sock).
Most UDP encapsulations only use kernel sockets though. It seems that
only L2TP and GTP use setup_udp_tunnel_sock() with userpsace sockets.
So it might be feasible to restrict usage of sk_user_data to kernel
sockets only.
For L2TP, we probably can adapt l2tp_sock_to_tunnel() so that it does
a lookup in a hashtable indexed by the socket pointer, rather than
dereferencing sk_user_data. That doesn't look very satisfying to me,
but that's the only way I found so far.
L2TP needs a way to get at its local data from the socket in the data path.
We also have another user of sk_user_data in l2tp_ppp, but since it
uses its own socket type, I guess we could simply embed the pointer in
its parent structure.
quoted
It looks like over time we've accumulated this new class of uses
of sk->sk_user_data, ho hum...
And it's not like we can add a test to KCM to avoid these socket
types, because they will look like normal UDP datagram sockets.
What a mess...
Furthermore, even if you add a test to KCM, you will now need to
add the same test to L2TP and anything else which uses sk_user_data
for tunneling and for which userspace has access to the socket fd.
And it will be racy, indeed, until all such users align to the same
precise locking scheme for tests and updates to sk_user_data.
Again, what a mess...
So, if I understand correctly, we can either restrict sk_user_data to
kernel sockets so that KCM couldn't act on them (but then why would we
make an exception for KCM and allow it to set sk_user_data on
non-kernel sockets?).
Or we could agree on a locking scheme for sk_user_data and update all
users so that they'd fail instead of overwriting it when it's not NULL.
Assuming my understanding is correct, do you have any preference for
fixing this issue? Or any other ideas?
Could we add a new pointer, say, encap_user_data to struct udp_sock
and use it instead of sk_user_data for UDP-encap sockets?
On Thu, Jan 18, 2018 at 03:40:52PM +0000, James Chapman wrote:
On 18 January 2018 at 15:18, Guillaume Nault [off-list ref] wrote:
quoted
On Wed, Jan 17, 2018 at 02:25:38PM -0500, David Miller wrote:
quoted
If all else was equal, even though it doesn't make much sense to KCM
attach L2TP sockets to KCM, I would suggest to change L2TP to store
it's private stuff elsewhere.
But that is not the case. Anything using the generic UDP
encapsulation layer is going to make use of sk->sk_user_data like this
(see setup_udp_tunnel_sock).
Most UDP encapsulations only use kernel sockets though. It seems that
only L2TP and GTP use setup_udp_tunnel_sock() with userpsace sockets.
So it might be feasible to restrict usage of sk_user_data to kernel
sockets only.
For L2TP, we probably can adapt l2tp_sock_to_tunnel() so that it does
a lookup in a hashtable indexed by the socket pointer, rather than
dereferencing sk_user_data. That doesn't look very satisfying to me,
but that's the only way I found so far.
L2TP needs a way to get at its local data from the socket in the data path.
Did I miss something? On xmit, the session is provided by l2tp_ppp or
l2tp_eth, which is enough to get access to the parent tunnel.
For reception, l2tp_udp_encap_recv() receives the socket pointer as
parameter and could get enough information from the headers to retrieve the
tunnel structure anymay (l2tp_ip and l2tp_ip6 use the headers).
l2tp_ppp also uses sk_user_data for its PPPOX sockets, but we probably
can handle this case more easily.
From: James Chapman <jchapman@katalix.com> Date: 2018-01-18 17:30:22
On 18 January 2018 at 16:29, Guillaume Nault [off-list ref] wrote:
On Thu, Jan 18, 2018 at 03:40:52PM +0000, James Chapman wrote:
quoted
On 18 January 2018 at 15:18, Guillaume Nault [off-list ref] wrote:
quoted
On Wed, Jan 17, 2018 at 02:25:38PM -0500, David Miller wrote:
quoted
If all else was equal, even though it doesn't make much sense to KCM
attach L2TP sockets to KCM, I would suggest to change L2TP to store
it's private stuff elsewhere.
But that is not the case. Anything using the generic UDP
encapsulation layer is going to make use of sk->sk_user_data like this
(see setup_udp_tunnel_sock).
Most UDP encapsulations only use kernel sockets though. It seems that
only L2TP and GTP use setup_udp_tunnel_sock() with userpsace sockets.
So it might be feasible to restrict usage of sk_user_data to kernel
sockets only.
For L2TP, we probably can adapt l2tp_sock_to_tunnel() so that it does
a lookup in a hashtable indexed by the socket pointer, rather than
dereferencing sk_user_data. That doesn't look very satisfying to me,
but that's the only way I found so far.
L2TP needs a way to get at its local data from the socket in the data path.
Did I miss something? On xmit, the session is provided by l2tp_ppp or
l2tp_eth, which is enough to get access to the parent tunnel.
For reception, l2tp_udp_encap_recv() receives the socket pointer as
parameter and could get enough information from the headers to retrieve the
tunnel structure anymay (l2tp_ip and l2tp_ip6 use the headers).
It's the receive side I was thinking about. It would be a little more
involved to derive the tunnel and session from the packet with UDP
since we'd have to handle L2TPv2 and L2TPv3.
From: Tom Herbert <hidden> Date: 2018-01-18 17:40:46
On Wed, Jan 17, 2018 at 11:25 AM, David Miller [off-list ref] wrote:
From: James Chapman <jchapman@katalix.com>
Date: Wed, 17 Jan 2018 11:13:33 +0000
quoted
On 16 January 2018 at 19:00, David Miller [off-list ref] wrote:
quoted
From: Tom Herbert <redacted>
Date: Tue, 16 Jan 2018 09:36:41 -0800
quoted
sk_user_data is set with the sk_callback lock held in code below.
Should be able to take the lock earlier can do this check under the
lock.
csock, and this csk, is obtained from an arbitrary one of the
process's FDs. It can be any socket type or family, and that socket's
family might set sk_user_data without the callback lock.
The only socket type check is making sure it is not another PF_KCM
socket. So that doesn't help with this problem.
Is it the intention to update all socket code over time to write
sk_user_data within the sk_callback lock? If so, I'm happy to address
that in the l2tp code (and update the kcm patch to check sk_user_data
within the sk_callback lock). Or is the preferred solution to restrict
KCM to specific socket families, as suggested by Guillaume earlier in
the thread?
I think we have a more fundamental issue here.
sk->sk_user_data is a place where RPC layer specific data is hung off
of. By this definition SunRPC, RXRPC, RDS, TIPC, and KCM are all
using it correctly.
Phonet has a similar issue to the one seen here, it tests and changes
sk_user_data under lock_sock(). The only requirement it makes is
that the socket type is not SOCK_STREAM. However, this one might be OK
since only pep_sock sockets can be passed down into gprs_attach().
Most of these cases like SunRPC, RXRPC, etc. are fine because they
only graft on top of TCP and UDP sockets.
The weird situation here is that L2TP does tunneling and stores it's
private state in sk->sk_user_data like an RPC layer would. And KCM
allows basically any socket type to be attached.
The RPC layers create their sockets internally, so I cannot see a way
that those can be sent to a KCM attach operations. And I think that
is why this RPC invariant is important for sk_user_data usage.
If all else was equal, even though it doesn't make much sense to KCM
attach L2TP sockets to KCM, I would suggest to change L2TP to store
it's private stuff elsewhere.
But that is not the case. Anything using the generic UDP
encapsulation layer is going to make use of sk->sk_user_data like this
(see setup_udp_tunnel_sock).
It looks like over time we've accumulated this new class of uses
of sk->sk_user_data, ho hum...
And it's not like we can add a test to KCM to avoid these socket
types, because they will look like normal UDP datagram sockets.
What a mess...
Furthermore, even if you add a test to KCM, you will now need to
add the same test to L2TP and anything else which uses sk_user_data
for tunneling and for which userspace has access to the socket fd.
And it will be racy, indeed, until all such users align to the same
precise locking scheme for tests and updates to sk_user_data.
Again, what a mess...
It's not so surprising that sk_user_data is being used for so many
purposes, it's quite a powerful and useful notion. So, to a large
extent I think it's a victim of it's own success.
Aligning to one locking scheme is the first task to clean this. The
second would be how to deal with multiple simulataneous use on a
socket (or maybe not allow). I've thought about having a chain of
sk_user_data, but that's only useful is the write/read callback are
also chained. All this starts to look like STREAMS at some point ;-)
Tom
From: Tom Herbert <hidden> Date: 2018-01-18 17:46:58
On Thu, Jan 18, 2018 at 7:40 AM, James Chapman [off-list ref] wrote:
On 18 January 2018 at 15:18, Guillaume Nault [off-list ref] wrote:
quoted
On Wed, Jan 17, 2018 at 02:25:38PM -0500, David Miller wrote:
quoted
From: James Chapman <jchapman@katalix.com>
Date: Wed, 17 Jan 2018 11:13:33 +0000
quoted
On 16 January 2018 at 19:00, David Miller [off-list ref] wrote:
quoted
From: Tom Herbert <redacted>
Date: Tue, 16 Jan 2018 09:36:41 -0800
quoted
sk_user_data is set with the sk_callback lock held in code below.
Should be able to take the lock earlier can do this check under the
lock.
csock, and this csk, is obtained from an arbitrary one of the
process's FDs. It can be any socket type or family, and that socket's
family might set sk_user_data without the callback lock.
The only socket type check is making sure it is not another PF_KCM
socket. So that doesn't help with this problem.
Is it the intention to update all socket code over time to write
sk_user_data within the sk_callback lock? If so, I'm happy to address
that in the l2tp code (and update the kcm patch to check sk_user_data
within the sk_callback lock). Or is the preferred solution to restrict
KCM to specific socket families, as suggested by Guillaume earlier in
the thread?
I think we have a more fundamental issue here.
sk->sk_user_data is a place where RPC layer specific data is hung off
of. By this definition SunRPC, RXRPC, RDS, TIPC, and KCM are all
using it correctly.
Phonet has a similar issue to the one seen here, it tests and changes
sk_user_data under lock_sock(). The only requirement it makes is
that the socket type is not SOCK_STREAM. However, this one might be OK
since only pep_sock sockets can be passed down into gprs_attach().
But, if I read it correctly, that doesn't prevent it from being passed
to kcm_attach() later on, which will overwrite sk_user_data (unless we
update the locking scheme and refuse to overwrite sk_user_data in a
race-free way).
BTW couldn't the gprs_dev pointer be embedded in struct pep_sock?
This way pep_sk(sk)->gp could be used instead of sk->sk_user_data.
That'd probably be a violation of the phonet's layering, as that'd
tie gprs_dev to pep sockets. OTOH, only pep sockets can currently be
attached to gprs_dev, so in practice that might be a reasonable
compromise.
quoted
Most of these cases like SunRPC, RXRPC, etc. are fine because they
only graft on top of TCP and UDP sockets.
The weird situation here is that L2TP does tunneling and stores it's
private state in sk->sk_user_data like an RPC layer would. And KCM
allows basically any socket type to be attached.
The RPC layers create their sockets internally, so I cannot see a way
that those can be sent to a KCM attach operations. And I think that
is why this RPC invariant is important for sk_user_data usage.
SunRPC seems to possibly set sk_user_data on user sockets: svc_addsock()
gets a socket using sockfd_lookup() then passes it to svc_setup_socket()
which in turn sets sk_user_data. I don't know anything about SunRPC, so
I might very well have missed important details, but I believe such a
socket could be passed to KCM which could lead to the same kind of
issues as for L2TP. Other RPCs look safe to me.
quoted
If all else was equal, even though it doesn't make much sense to KCM
attach L2TP sockets to KCM, I would suggest to change L2TP to store
it's private stuff elsewhere.
But that is not the case. Anything using the generic UDP
encapsulation layer is going to make use of sk->sk_user_data like this
(see setup_udp_tunnel_sock).
Most UDP encapsulations only use kernel sockets though. It seems that
only L2TP and GTP use setup_udp_tunnel_sock() with userpsace sockets.
So it might be feasible to restrict usage of sk_user_data to kernel
sockets only.
For L2TP, we probably can adapt l2tp_sock_to_tunnel() so that it does
a lookup in a hashtable indexed by the socket pointer, rather than
dereferencing sk_user_data. That doesn't look very satisfying to me,
but that's the only way I found so far.
L2TP needs a way to get at its local data from the socket in the data path.
quoted
We also have another user of sk_user_data in l2tp_ppp, but since it
uses its own socket type, I guess we could simply embed the pointer in
its parent structure.
quoted
It looks like over time we've accumulated this new class of uses
of sk->sk_user_data, ho hum...
And it's not like we can add a test to KCM to avoid these socket
types, because they will look like normal UDP datagram sockets.
What a mess...
Furthermore, even if you add a test to KCM, you will now need to
add the same test to L2TP and anything else which uses sk_user_data
for tunneling and for which userspace has access to the socket fd.
And it will be racy, indeed, until all such users align to the same
precise locking scheme for tests and updates to sk_user_data.
Again, what a mess...
So, if I understand correctly, we can either restrict sk_user_data to
kernel sockets so that KCM couldn't act on them (but then why would we
make an exception for KCM and allow it to set sk_user_data on
non-kernel sockets?).
Or we could agree on a locking scheme for sk_user_data and update all
users so that they'd fail instead of overwriting it when it's not NULL.
Assuming my understanding is correct, do you have any preference for
fixing this issue? Or any other ideas?
Could we add a new pointer, say, encap_user_data to struct udp_sock
and use it instead of sk_user_data for UDP-encap sockets?
Then that's increasing the udp_sock structure size for a narrow use
case which will get push back. I think it's going to be better to
stick with one sock pointer. We could maybe redefine sk_user_data as a
pointer to an allocated structure or array so it can hold multiple
user_data pointers (in lieu of chaining).
Tom
From: Eric Dumazet <hidden> Date: 2018-01-18 18:08:15
On Thu, 2018-01-18 at 09:46 -0800, Tom Herbert wrote:
Then that's increasing the udp_sock structure size for a narrow use
case which will get push back. I think it's going to be better to
stick with one sock pointer. We could maybe redefine sk_user_data as a
pointer to an allocated structure or array so it can hold multiple
user_data pointers (in lieu of chaining).
We do not have a lot of UDP sockets per host, I do not believe it
should be a problem adding stuff in them.
From: Tom Herbert <hidden> Date: 2018-01-18 19:26:58
On Thu, Jan 18, 2018 at 10:08 AM, Eric Dumazet [off-list ref] wrote:
On Thu, 2018-01-18 at 09:46 -0800, Tom Herbert wrote:
quoted
Then that's increasing the udp_sock structure size for a narrow use
case which will get push back. I think it's going to be better to
stick with one sock pointer. We could maybe redefine sk_user_data as a
pointer to an allocated structure or array so it can hold multiple
user_data pointers (in lieu of chaining).
We do not have a lot of UDP sockets per host, I do not believe it
should be a problem adding stuff in them.
From: Eric Dumazet <hidden> Date: 2018-01-18 19:46:35
On Thu, 2018-01-18 at 11:26 -0800, Tom Herbert wrote:
On Thu, Jan 18, 2018 at 10:08 AM, Eric Dumazet [off-list ref] wrote:
quoted
On Thu, 2018-01-18 at 09:46 -0800, Tom Herbert wrote:
quoted
Then that's increasing the udp_sock structure size for a narrow use
case which will get push back. I think it's going to be better to
stick with one sock pointer. We could maybe redefine sk_user_data as a
pointer to an allocated structure or array so it can hold multiple
user_data pointers (in lieu of chaining).
We do not have a lot of UDP sockets per host, I do not believe it
should be a problem adding stuff in them.
Eric,
Is QUIC using unconnected sockets then?
Server side is using a bunch of unconnected sockets, usually one per
cpu.
Note that UDP stack has no 4-tuple proper support yet, and even if it
had, extra memory costs would be huge on servers handling millions of
flows.