[PATCH] Bluetooth: fix use-after-free error in lock_sock_nested()

Subsystems: bluetooth subsystem, the rest

STALE1863d

6 messages, 2 authors, 2021-07-19 · open the first message on its own page

[PATCH] Bluetooth: fix use-after-free error in lock_sock_nested()

From: Wang ShaoBo <hidden>
Date: 2021-07-14 03:21:10

use-after-free error in lock_sock_nested() is reported:

[  179.140137][ T3731] =====================================================
[  179.142675][ T3731] BUG: KMSAN: use-after-free in lock_sock_nested+0x280/0x2c0
[  179.145494][ T3731] CPU: 4 PID: 3731 Comm: kworker/4:2 Not tainted 5.12.0-rc6+ #54
[  179.148432][ T3731] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
[  179.151806][ T3731] Workqueue: events l2cap_chan_timeout
[  179.152730][ T3731] Call Trace:
[  179.153301][ T3731]  dump_stack+0x24c/0x2e0
[  179.154063][ T3731]  kmsan_report+0xfb/0x1e0
[  179.154855][ T3731]  __msan_warning+0x5c/0xa0
[  179.155579][ T3731]  lock_sock_nested+0x280/0x2c0
[  179.156436][ T3731]  ? kmsan_get_metadata+0x116/0x180
[  179.157257][ T3731]  l2cap_sock_teardown_cb+0xb8/0x890
[  179.158154][ T3731]  ? __msan_metadata_ptr_for_load_8+0x10/0x20
[  179.159141][ T3731]  ? kmsan_get_metadata+0x116/0x180
[  179.159994][ T3731]  ? kmsan_get_shadow_origin_ptr+0x84/0xb0
[  179.160959][ T3731]  ? l2cap_sock_recv_cb+0x420/0x420
[  179.161834][ T3731]  l2cap_chan_del+0x3e1/0x1d50
[  179.162608][ T3731]  ? kmsan_get_metadata+0x116/0x180
[  179.163435][ T3731]  ? kmsan_get_shadow_origin_ptr+0x84/0xb0
[  179.164406][ T3731]  l2cap_chan_close+0xeea/0x1050
[  179.165189][ T3731]  ? kmsan_internal_unpoison_shadow+0x42/0x70
[  179.166180][ T3731]  l2cap_chan_timeout+0x1da/0x590
[  179.167066][ T3731]  ? __msan_metadata_ptr_for_load_8+0x10/0x20
[  179.168023][ T3731]  ? l2cap_chan_create+0x560/0x560
[  179.168818][ T3731]  process_one_work+0x121d/0x1ff0
[  179.169598][ T3731]  worker_thread+0x121b/0x2370
[  179.170346][ T3731]  kthread+0x4ef/0x610
[  179.171010][ T3731]  ? process_one_work+0x1ff0/0x1ff0
[  179.171828][ T3731]  ? kthread_blkcg+0x110/0x110
[  179.172587][ T3731]  ret_from_fork+0x1f/0x30
[  179.173348][ T3731]
[  179.173752][ T3731] Uninit was created at:
[  179.174409][ T3731]  kmsan_internal_poison_shadow+0x5c/0xf0
[  179.175373][ T3731]  kmsan_slab_free+0x76/0xc0
[  179.176060][ T3731]  kfree+0x3a5/0x1180
[  179.176664][ T3731]  __sk_destruct+0x8af/0xb80
[  179.177375][ T3731]  __sk_free+0x812/0x8c0
[  179.178032][ T3731]  sk_free+0x97/0x130
[  179.178686][ T3731]  l2cap_sock_release+0x3d5/0x4d0
[  179.179457][ T3731]  sock_close+0x150/0x450
[  179.180117][ T3731]  __fput+0x6bd/0xf00
[  179.180787][ T3731]  ____fput+0x37/0x40
[  179.181481][ T3731]  task_work_run+0x140/0x280
[  179.182219][ T3731]  do_exit+0xe51/0x3e60
[  179.182930][ T3731]  do_group_exit+0x20e/0x450
[  179.183656][ T3731]  get_signal+0x2dfb/0x38f0
[  179.184344][ T3731]  arch_do_signal_or_restart+0xaa/0xe10
[  179.185266][ T3731]  exit_to_user_mode_prepare+0x2d2/0x560
[  179.186136][ T3731]  syscall_exit_to_user_mode+0x35/0x60
[  179.186984][ T3731]  do_syscall_64+0xc5/0x140
[  179.187681][ T3731]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[  179.188604][ T3731] =====================================================

In our case, there are two Thread A and B:

Context: Thread A:              Context: Thread B:

l2cap_chan_timeout()            __se_sys_shutdown()
  l2cap_chan_close()              l2cap_sock_shutdown()
    l2cap_chan_del()                l2cap_chan_close()
      l2cap_sock_teardown_cb()        l2cap_sock_teardown_cb()

Once l2cap_sock_teardown_cb() excuted, this sock will be marked as SOCK_ZAPPED,
and can be treated as killable in l2cap_sock_kill() if sock_orphan() has
excuted, at this time we close sock through sock_close() which end to call
l2cap_sock_kill() like Thread C:

Context: Thread C:

sock_close()
  l2cap_sock_release()
    sock_orphan()
    l2cap_sock_kill()  #free sock if refcnt is 1

If C completed, Once A or B reaches l2cap_sock_teardown_cb() again,
use-after-free happened.

We should set chan->data to NULL if sock is freed, for telling teardown
operation is not allowed in l2cap_sock_teardown_cb(), and also we should
avoid killing an already killed socket in l2cap_sock_close_cb().

Signed-off-by: Wang ShaoBo <redacted>
---
 net/bluetooth/l2cap_sock.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/net/bluetooth/l2cap_sock.c b/net/bluetooth/l2cap_sock.c
index c99d65ef13b1..ddc6a692b237 100644
--- a/net/bluetooth/l2cap_sock.c
+++ b/net/bluetooth/l2cap_sock.c
@@ -1215,14 +1215,18 @@ static int l2cap_sock_recvmsg(struct socket *sock, struct msghdr *msg,
  */
 static void l2cap_sock_kill(struct sock *sk)
 {
+	struct l2cap_chan *chan;
+
 	if (!sock_flag(sk, SOCK_ZAPPED) || sk->sk_socket)
 		return;
 
 	BT_DBG("sk %p state %s", sk, state_to_string(sk->sk_state));
 
 	/* Kill poor orphan */
-
-	l2cap_chan_put(l2cap_pi(sk)->chan);
+	chan = l2cap_pi(sk)->chan;
+	l2cap_chan_put(chan);
+	if (refcount_read(&sk->sk_refcnt) == 1)
+		chan->data = NULL;
 	sock_set_flag(sk, SOCK_DEAD);
 	sock_put(sk);
 }
@@ -1508,6 +1512,9 @@ static void l2cap_sock_close_cb(struct l2cap_chan *chan)
 {
 	struct sock *sk = chan->data;
 
+	if (!sk)
+		return;
+
 	l2cap_sock_kill(sk);
 }
 
@@ -1516,6 +1523,9 @@ static void l2cap_sock_teardown_cb(struct l2cap_chan *chan, int err)
 	struct sock *sk = chan->data;
 	struct sock *parent;
 
+	if (!sk)
+		return;
+
 	BT_DBG("chan %p state %s", chan, state_to_string(chan->state));
 
 	/* This callback can be called both for server (BT_LISTEN)
-- 
2.27.0

Re: [PATCH] Bluetooth: fix use-after-free error in lock_sock_nested()

From: Luiz Augusto von Dentz <luiz.dentz@gmail.com>
Date: 2021-07-14 21:50:23

Hi,

On Tue, Jul 13, 2021 at 8:20 PM Wang ShaoBo [off-list ref] wrote:
quoted hunk
use-after-free error in lock_sock_nested() is reported:

[  179.140137][ T3731] =====================================================
[  179.142675][ T3731] BUG: KMSAN: use-after-free in lock_sock_nested+0x280/0x2c0
[  179.145494][ T3731] CPU: 4 PID: 3731 Comm: kworker/4:2 Not tainted 5.12.0-rc6+ #54
[  179.148432][ T3731] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
[  179.151806][ T3731] Workqueue: events l2cap_chan_timeout
[  179.152730][ T3731] Call Trace:
[  179.153301][ T3731]  dump_stack+0x24c/0x2e0
[  179.154063][ T3731]  kmsan_report+0xfb/0x1e0
[  179.154855][ T3731]  __msan_warning+0x5c/0xa0
[  179.155579][ T3731]  lock_sock_nested+0x280/0x2c0
[  179.156436][ T3731]  ? kmsan_get_metadata+0x116/0x180
[  179.157257][ T3731]  l2cap_sock_teardown_cb+0xb8/0x890
[  179.158154][ T3731]  ? __msan_metadata_ptr_for_load_8+0x10/0x20
[  179.159141][ T3731]  ? kmsan_get_metadata+0x116/0x180
[  179.159994][ T3731]  ? kmsan_get_shadow_origin_ptr+0x84/0xb0
[  179.160959][ T3731]  ? l2cap_sock_recv_cb+0x420/0x420
[  179.161834][ T3731]  l2cap_chan_del+0x3e1/0x1d50
[  179.162608][ T3731]  ? kmsan_get_metadata+0x116/0x180
[  179.163435][ T3731]  ? kmsan_get_shadow_origin_ptr+0x84/0xb0
[  179.164406][ T3731]  l2cap_chan_close+0xeea/0x1050
[  179.165189][ T3731]  ? kmsan_internal_unpoison_shadow+0x42/0x70
[  179.166180][ T3731]  l2cap_chan_timeout+0x1da/0x590
[  179.167066][ T3731]  ? __msan_metadata_ptr_for_load_8+0x10/0x20
[  179.168023][ T3731]  ? l2cap_chan_create+0x560/0x560
[  179.168818][ T3731]  process_one_work+0x121d/0x1ff0
[  179.169598][ T3731]  worker_thread+0x121b/0x2370
[  179.170346][ T3731]  kthread+0x4ef/0x610
[  179.171010][ T3731]  ? process_one_work+0x1ff0/0x1ff0
[  179.171828][ T3731]  ? kthread_blkcg+0x110/0x110
[  179.172587][ T3731]  ret_from_fork+0x1f/0x30
[  179.173348][ T3731]
[  179.173752][ T3731] Uninit was created at:
[  179.174409][ T3731]  kmsan_internal_poison_shadow+0x5c/0xf0
[  179.175373][ T3731]  kmsan_slab_free+0x76/0xc0
[  179.176060][ T3731]  kfree+0x3a5/0x1180
[  179.176664][ T3731]  __sk_destruct+0x8af/0xb80
[  179.177375][ T3731]  __sk_free+0x812/0x8c0
[  179.178032][ T3731]  sk_free+0x97/0x130
[  179.178686][ T3731]  l2cap_sock_release+0x3d5/0x4d0
[  179.179457][ T3731]  sock_close+0x150/0x450
[  179.180117][ T3731]  __fput+0x6bd/0xf00
[  179.180787][ T3731]  ____fput+0x37/0x40
[  179.181481][ T3731]  task_work_run+0x140/0x280
[  179.182219][ T3731]  do_exit+0xe51/0x3e60
[  179.182930][ T3731]  do_group_exit+0x20e/0x450
[  179.183656][ T3731]  get_signal+0x2dfb/0x38f0
[  179.184344][ T3731]  arch_do_signal_or_restart+0xaa/0xe10
[  179.185266][ T3731]  exit_to_user_mode_prepare+0x2d2/0x560
[  179.186136][ T3731]  syscall_exit_to_user_mode+0x35/0x60
[  179.186984][ T3731]  do_syscall_64+0xc5/0x140
[  179.187681][ T3731]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[  179.188604][ T3731] =====================================================

In our case, there are two Thread A and B:

Context: Thread A:              Context: Thread B:

l2cap_chan_timeout()            __se_sys_shutdown()
  l2cap_chan_close()              l2cap_sock_shutdown()
    l2cap_chan_del()                l2cap_chan_close()
      l2cap_sock_teardown_cb()        l2cap_sock_teardown_cb()

Once l2cap_sock_teardown_cb() excuted, this sock will be marked as SOCK_ZAPPED,
and can be treated as killable in l2cap_sock_kill() if sock_orphan() has
excuted, at this time we close sock through sock_close() which end to call
l2cap_sock_kill() like Thread C:

Context: Thread C:

sock_close()
  l2cap_sock_release()
    sock_orphan()
    l2cap_sock_kill()  #free sock if refcnt is 1

If C completed, Once A or B reaches l2cap_sock_teardown_cb() again,
use-after-free happened.

We should set chan->data to NULL if sock is freed, for telling teardown
operation is not allowed in l2cap_sock_teardown_cb(), and also we should
avoid killing an already killed socket in l2cap_sock_close_cb().

Signed-off-by: Wang ShaoBo <redacted>
---
 net/bluetooth/l2cap_sock.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/net/bluetooth/l2cap_sock.c b/net/bluetooth/l2cap_sock.c
index c99d65ef13b1..ddc6a692b237 100644
--- a/net/bluetooth/l2cap_sock.c
+++ b/net/bluetooth/l2cap_sock.c
@@ -1215,14 +1215,18 @@ static int l2cap_sock_recvmsg(struct socket *sock, struct msghdr *msg,
  */
 static void l2cap_sock_kill(struct sock *sk)
 {
+       struct l2cap_chan *chan;
+
        if (!sock_flag(sk, SOCK_ZAPPED) || sk->sk_socket)
                return;

        BT_DBG("sk %p state %s", sk, state_to_string(sk->sk_state));

        /* Kill poor orphan */
-
-       l2cap_chan_put(l2cap_pi(sk)->chan);
+       chan = l2cap_pi(sk)->chan;
+       l2cap_chan_put(chan);
+       if (refcount_read(&sk->sk_refcnt) == 1)
+               chan->data = NULL;
Instead of checking if it is the last reference here, wouldn't it be
better to reset the chan->data to NULL on l2cap_sock_destruct?
quoted hunk
        sock_set_flag(sk, SOCK_DEAD);
        sock_put(sk);
 }
@@ -1508,6 +1512,9 @@ static void l2cap_sock_close_cb(struct l2cap_chan *chan)
 {
        struct sock *sk = chan->data;

+       if (!sk)
+               return;
+
        l2cap_sock_kill(sk);
 }
@@ -1516,6 +1523,9 @@ static void l2cap_sock_teardown_cb(struct l2cap_chan *chan, int err)
        struct sock *sk = chan->data;
        struct sock *parent;

+       if (!sk)
+               return;
+
        BT_DBG("chan %p state %s", chan, state_to_string(chan->state));

        /* This callback can be called both for server (BT_LISTEN)
--
2.27.0

-- 
Luiz Augusto von Dentz

Re: [PATCH] Bluetooth: fix use-after-free error in lock_sock_nested()

From: Wangshaobo (bobo) <hidden>
Date: 2021-07-15 04:58:00

在 2021/7/15 5:50, Luiz Augusto von Dentz 写道:
Hi,

On Tue, Jul 13, 2021 at 8:20 PM Wang ShaoBo [off-list ref] wrote:
quoted
use-after-free error in lock_sock_nested() is reported:

[  179.140137][ T3731] =====================================================
[  179.142675][ T3731] BUG: KMSAN: use-after-free in lock_sock_nested+0x280/0x2c0
[  179.145494][ T3731] CPU: 4 PID: 3731 Comm: kworker/4:2 Not tainted 5.12.0-rc6+ #54
[  179.148432][ T3731] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
[  179.151806][ T3731] Workqueue: events l2cap_chan_timeout
[  179.152730][ T3731] Call Trace:
[  179.153301][ T3731]  dump_stack+0x24c/0x2e0
[  179.154063][ T3731]  kmsan_report+0xfb/0x1e0
[  179.154855][ T3731]  __msan_warning+0x5c/0xa0
[  179.155579][ T3731]  lock_sock_nested+0x280/0x2c0
[  179.156436][ T3731]  ? kmsan_get_metadata+0x116/0x180
[  179.157257][ T3731]  l2cap_sock_teardown_cb+0xb8/0x890
[  179.158154][ T3731]  ? __msan_metadata_ptr_for_load_8+0x10/0x20
[  179.159141][ T3731]  ? kmsan_get_metadata+0x116/0x180
[  179.159994][ T3731]  ? kmsan_get_shadow_origin_ptr+0x84/0xb0
[  179.160959][ T3731]  ? l2cap_sock_recv_cb+0x420/0x420
[  179.161834][ T3731]  l2cap_chan_del+0x3e1/0x1d50
[  179.162608][ T3731]  ? kmsan_get_metadata+0x116/0x180
[  179.163435][ T3731]  ? kmsan_get_shadow_origin_ptr+0x84/0xb0
[  179.164406][ T3731]  l2cap_chan_close+0xeea/0x1050
[  179.165189][ T3731]  ? kmsan_internal_unpoison_shadow+0x42/0x70
[  179.166180][ T3731]  l2cap_chan_timeout+0x1da/0x590
[  179.167066][ T3731]  ? __msan_metadata_ptr_for_load_8+0x10/0x20
[  179.168023][ T3731]  ? l2cap_chan_create+0x560/0x560
[  179.168818][ T3731]  process_one_work+0x121d/0x1ff0
[  179.169598][ T3731]  worker_thread+0x121b/0x2370
[  179.170346][ T3731]  kthread+0x4ef/0x610
[  179.171010][ T3731]  ? process_one_work+0x1ff0/0x1ff0
[  179.171828][ T3731]  ? kthread_blkcg+0x110/0x110
[  179.172587][ T3731]  ret_from_fork+0x1f/0x30
[  179.173348][ T3731]
[  179.173752][ T3731] Uninit was created at:
[  179.174409][ T3731]  kmsan_internal_poison_shadow+0x5c/0xf0
[  179.175373][ T3731]  kmsan_slab_free+0x76/0xc0
[  179.176060][ T3731]  kfree+0x3a5/0x1180
[  179.176664][ T3731]  __sk_destruct+0x8af/0xb80
[  179.177375][ T3731]  __sk_free+0x812/0x8c0
[  179.178032][ T3731]  sk_free+0x97/0x130
[  179.178686][ T3731]  l2cap_sock_release+0x3d5/0x4d0
[  179.179457][ T3731]  sock_close+0x150/0x450
[  179.180117][ T3731]  __fput+0x6bd/0xf00
[  179.180787][ T3731]  ____fput+0x37/0x40
[  179.181481][ T3731]  task_work_run+0x140/0x280
[  179.182219][ T3731]  do_exit+0xe51/0x3e60
[  179.182930][ T3731]  do_group_exit+0x20e/0x450
[  179.183656][ T3731]  get_signal+0x2dfb/0x38f0
[  179.184344][ T3731]  arch_do_signal_or_restart+0xaa/0xe10
[  179.185266][ T3731]  exit_to_user_mode_prepare+0x2d2/0x560
[  179.186136][ T3731]  syscall_exit_to_user_mode+0x35/0x60
[  179.186984][ T3731]  do_syscall_64+0xc5/0x140
[  179.187681][ T3731]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[  179.188604][ T3731] =====================================================

In our case, there are two Thread A and B:

Context: Thread A:              Context: Thread B:

l2cap_chan_timeout()            __se_sys_shutdown()
   l2cap_chan_close()              l2cap_sock_shutdown()
     l2cap_chan_del()                l2cap_chan_close()
       l2cap_sock_teardown_cb()        l2cap_sock_teardown_cb()

Once l2cap_sock_teardown_cb() excuted, this sock will be marked as SOCK_ZAPPED,
and can be treated as killable in l2cap_sock_kill() if sock_orphan() has
excuted, at this time we close sock through sock_close() which end to call
l2cap_sock_kill() like Thread C:

Context: Thread C:

sock_close()
   l2cap_sock_release()
     sock_orphan()
     l2cap_sock_kill()  #free sock if refcnt is 1

If C completed, Once A or B reaches l2cap_sock_teardown_cb() again,
use-after-free happened.

We should set chan->data to NULL if sock is freed, for telling teardown
operation is not allowed in l2cap_sock_teardown_cb(), and also we should
avoid killing an already killed socket in l2cap_sock_close_cb().

Signed-off-by: Wang ShaoBo <redacted>
---
  net/bluetooth/l2cap_sock.c | 14 ++++++++++++--
  1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/net/bluetooth/l2cap_sock.c b/net/bluetooth/l2cap_sock.c
index c99d65ef13b1..ddc6a692b237 100644
--- a/net/bluetooth/l2cap_sock.c
+++ b/net/bluetooth/l2cap_sock.c
@@ -1215,14 +1215,18 @@ static int l2cap_sock_recvmsg(struct socket *sock, struct msghdr *msg,
   */
  static void l2cap_sock_kill(struct sock *sk)
  {
+       struct l2cap_chan *chan;
+
         if (!sock_flag(sk, SOCK_ZAPPED) || sk->sk_socket)
                 return;

         BT_DBG("sk %p state %s", sk, state_to_string(sk->sk_state));

         /* Kill poor orphan */
-
-       l2cap_chan_put(l2cap_pi(sk)->chan);
+       chan = l2cap_pi(sk)->chan;
+       l2cap_chan_put(chan);
There is a problem here, the above sentence `l2cap_chan_put(chan)` 
should put after

following sentence.
quoted
+       if (refcount_read(&sk->sk_refcnt) == 1)
+               chan->data = NULL;
Instead of checking if it is the last reference here, wouldn't it be
better to reset the chan->data to NULL on l2cap_sock_destruct?
Hi,

In my case it looks OK, this is the diff:
diff --git a/net/bluetooth/l2cap_sock.c b/net/bluetooth/l2cap_sock.c
index f1b1edd0b697..32ef3328ab49 100644
--- a/net/bluetooth/l2cap_sock.c
+++ b/net/bluetooth/l2cap_sock.c
@@ -1500,6 +1500,9 @@ static void l2cap_sock_close_cb(struct l2cap_chan 
*chan)
  {
         struct sock *sk = chan->data;

+       if (!sk)
+               return;
+
         l2cap_sock_kill(sk);
  }
@@ -1508,6 +1511,9 @@ static void l2cap_sock_teardown_cb(struct 
l2cap_chan *chan, int err)
         struct sock *sk = chan->data;
         struct sock *parent;

+       if (!sk)
+               return;
+
         BT_DBG("chan %p state %s", chan, state_to_string(chan->state));

         /* This callback can be called both for server (BT_LISTEN)
@@ -1700,6 +1706,7 @@ static void l2cap_sock_destruct(struct sock *sk)
         BT_DBG("sk %p", sk);

         if (l2cap_pi(sk)->chan)
+              l2cap_pi(sk)->chan->data = NULL;
                  l2cap_chan_put(l2cap_pi(sk)->chan);
But if it has potential risk if l2cap_sock_destruct() can not be excuted 
in time ?

sk_free():

         if (refcount_dec_and_test(&sk->sk_wmem_alloc)) //is possible 
this condition false ?

               __sk_free(sk)   -> ... l2cap_sock_destruct()

Re: [PATCH] Bluetooth: fix use-after-free error in lock_sock_nested()

From: Wangshaobo (bobo) <hidden>
Date: 2021-07-16 01:38:05

在 2021/7/15 12:57, Wangshaobo (bobo) 写道:
quoted hunk
在 2021/7/15 5:50, Luiz Augusto von Dentz 写道:
quoted
Hi,

On Tue, Jul 13, 2021 at 8:20 PM Wang ShaoBo 
[off-list ref] wrote:
quoted
use-after-free error in lock_sock_nested() is reported:

[  179.140137][ T3731] 
=====================================================
[  179.142675][ T3731] BUG: KMSAN: use-after-free in 
lock_sock_nested+0x280/0x2c0
[  179.145494][ T3731] CPU: 4 PID: 3731 Comm: kworker/4:2 Not 
tainted 5.12.0-rc6+ #54
[  179.148432][ T3731] Hardware name: QEMU Standard PC (i440FX + 
PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
[  179.151806][ T3731] Workqueue: events l2cap_chan_timeout
[  179.152730][ T3731] Call Trace:
[  179.153301][ T3731]  dump_stack+0x24c/0x2e0
[  179.154063][ T3731]  kmsan_report+0xfb/0x1e0
[  179.154855][ T3731]  __msan_warning+0x5c/0xa0
[  179.155579][ T3731]  lock_sock_nested+0x280/0x2c0
[  179.156436][ T3731]  ? kmsan_get_metadata+0x116/0x180
[  179.157257][ T3731]  l2cap_sock_teardown_cb+0xb8/0x890
[  179.158154][ T3731]  ? __msan_metadata_ptr_for_load_8+0x10/0x20
[  179.159141][ T3731]  ? kmsan_get_metadata+0x116/0x180
[  179.159994][ T3731]  ? kmsan_get_shadow_origin_ptr+0x84/0xb0
[  179.160959][ T3731]  ? l2cap_sock_recv_cb+0x420/0x420
[  179.161834][ T3731]  l2cap_chan_del+0x3e1/0x1d50
[  179.162608][ T3731]  ? kmsan_get_metadata+0x116/0x180
[  179.163435][ T3731]  ? kmsan_get_shadow_origin_ptr+0x84/0xb0
[  179.164406][ T3731]  l2cap_chan_close+0xeea/0x1050
[  179.165189][ T3731]  ? kmsan_internal_unpoison_shadow+0x42/0x70
[  179.166180][ T3731]  l2cap_chan_timeout+0x1da/0x590
[  179.167066][ T3731]  ? __msan_metadata_ptr_for_load_8+0x10/0x20
[  179.168023][ T3731]  ? l2cap_chan_create+0x560/0x560
[  179.168818][ T3731]  process_one_work+0x121d/0x1ff0
[  179.169598][ T3731]  worker_thread+0x121b/0x2370
[  179.170346][ T3731]  kthread+0x4ef/0x610
[  179.171010][ T3731]  ? process_one_work+0x1ff0/0x1ff0
[  179.171828][ T3731]  ? kthread_blkcg+0x110/0x110
[  179.172587][ T3731]  ret_from_fork+0x1f/0x30
[  179.173348][ T3731]
[  179.173752][ T3731] Uninit was created at:
[  179.174409][ T3731]  kmsan_internal_poison_shadow+0x5c/0xf0
[  179.175373][ T3731]  kmsan_slab_free+0x76/0xc0
[  179.176060][ T3731]  kfree+0x3a5/0x1180
[  179.176664][ T3731]  __sk_destruct+0x8af/0xb80
[  179.177375][ T3731]  __sk_free+0x812/0x8c0
[  179.178032][ T3731]  sk_free+0x97/0x130
[  179.178686][ T3731]  l2cap_sock_release+0x3d5/0x4d0
[  179.179457][ T3731]  sock_close+0x150/0x450
[  179.180117][ T3731]  __fput+0x6bd/0xf00
[  179.180787][ T3731]  ____fput+0x37/0x40
[  179.181481][ T3731]  task_work_run+0x140/0x280
[  179.182219][ T3731]  do_exit+0xe51/0x3e60
[  179.182930][ T3731]  do_group_exit+0x20e/0x450
[  179.183656][ T3731]  get_signal+0x2dfb/0x38f0
[  179.184344][ T3731]  arch_do_signal_or_restart+0xaa/0xe10
[  179.185266][ T3731]  exit_to_user_mode_prepare+0x2d2/0x560
[  179.186136][ T3731]  syscall_exit_to_user_mode+0x35/0x60
[  179.186984][ T3731]  do_syscall_64+0xc5/0x140
[  179.187681][ T3731] entry_SYSCALL_64_after_hwframe+0x44/0xae
[  179.188604][ T3731] 
=====================================================

In our case, there are two Thread A and B:

Context: Thread A:              Context: Thread B:

l2cap_chan_timeout()            __se_sys_shutdown()
   l2cap_chan_close()              l2cap_sock_shutdown()
     l2cap_chan_del()                l2cap_chan_close()
       l2cap_sock_teardown_cb() l2cap_sock_teardown_cb()

Once l2cap_sock_teardown_cb() excuted, this sock will be marked as 
SOCK_ZAPPED,
and can be treated as killable in l2cap_sock_kill() if sock_orphan() 
has
excuted, at this time we close sock through sock_close() which end 
to call
l2cap_sock_kill() like Thread C:

Context: Thread C:

sock_close()
   l2cap_sock_release()
     sock_orphan()
     l2cap_sock_kill()  #free sock if refcnt is 1

If C completed, Once A or B reaches l2cap_sock_teardown_cb() again,
use-after-free happened.

We should set chan->data to NULL if sock is freed, for telling teardown
operation is not allowed in l2cap_sock_teardown_cb(), and also we 
should
avoid killing an already killed socket in l2cap_sock_close_cb().

Signed-off-by: Wang ShaoBo <redacted>
---
  net/bluetooth/l2cap_sock.c | 14 ++++++++++++--
  1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/net/bluetooth/l2cap_sock.c b/net/bluetooth/l2cap_sock.c
index c99d65ef13b1..ddc6a692b237 100644
--- a/net/bluetooth/l2cap_sock.c
+++ b/net/bluetooth/l2cap_sock.c
@@ -1215,14 +1215,18 @@ static int l2cap_sock_recvmsg(struct socket 
*sock, struct msghdr *msg,
   */
  static void l2cap_sock_kill(struct sock *sk)
  {
+       struct l2cap_chan *chan;
+
         if (!sock_flag(sk, SOCK_ZAPPED) || sk->sk_socket)
                 return;

         BT_DBG("sk %p state %s", sk, state_to_string(sk->sk_state));

         /* Kill poor orphan */
-
-       l2cap_chan_put(l2cap_pi(sk)->chan);
+       chan = l2cap_pi(sk)->chan;
+       l2cap_chan_put(chan);
There is a problem here, the above sentence `l2cap_chan_put(chan)` 
should put after

following sentence.
quoted
quoted
+       if (refcount_read(&sk->sk_refcnt) == 1)
+               chan->data = NULL;
Instead of checking if it is the last reference here, wouldn't it be
better to reset the chan->data to NULL on l2cap_sock_destruct?
Hi,

In my case it looks OK, this is the diff:
diff --git a/net/bluetooth/l2cap_sock.c b/net/bluetooth/l2cap_sock.c
index f1b1edd0b697..32ef3328ab49 100644
--- a/net/bluetooth/l2cap_sock.c
+++ b/net/bluetooth/l2cap_sock.c
@@ -1500,6 +1500,9 @@ static void l2cap_sock_close_cb(struct 
l2cap_chan *chan)
 {
        struct sock *sk = chan->data;

+       if (!sk)
+               return;
+
        l2cap_sock_kill(sk);
 }
@@ -1508,6 +1511,9 @@ static void l2cap_sock_teardown_cb(struct 
l2cap_chan *chan, int err)
        struct sock *sk = chan->data;
        struct sock *parent;

+       if (!sk)
+               return;
+
        BT_DBG("chan %p state %s", chan, state_to_string(chan->state));

        /* This callback can be called both for server (BT_LISTEN)
@@ -1700,6 +1706,7 @@ static void l2cap_sock_destruct(struct sock *sk)
        BT_DBG("sk %p", sk);

        if (l2cap_pi(sk)->chan)
+              l2cap_pi(sk)->chan->data = NULL;
                 l2cap_chan_put(l2cap_pi(sk)->chan);

But if it has potential risk if l2cap_sock_destruct() can not be 
excuted in time ?

sk_free():

        if (refcount_dec_and_test(&sk->sk_wmem_alloc)) //is possible 
this condition false ?

              __sk_free(sk)   -> ... l2cap_sock_destruct()
Dear Luiz,

Not only that, if l2cap_sock_kill() has put 'l2cap_pi(sk)->chan', how 
does we avoid re-puting 'l2cap_pi(sk)->chan' if l2cap_sock_destruct() 
work postponed? this will cause underflow of chan->refcount; this PATCH 
4e1a720d0312 ("Bluetooth: avoid killing an already killed socket") also 
may not work in any case because only sock_orphan() has excuted can this 
sock be killed, but if sco_sock_release() excute first, for this sock 
has been marked as SOCK_DEAD, this sock can never be killed. So should 
we think put chan->data = NULL in xx_sock_kill() is a better choice ?

- WangShaoBo

Re: [PATCH] Bluetooth: fix use-after-free error in lock_sock_nested()

From: Luiz Augusto von Dentz <luiz.dentz@gmail.com>
Date: 2021-07-16 17:22:33

Hi,

On Thu, Jul 15, 2021 at 6:38 PM Wangshaobo (bobo)
[off-list ref] wrote:

在 2021/7/15 12:57, Wangshaobo (bobo) 写道:
quoted
在 2021/7/15 5:50, Luiz Augusto von Dentz 写道:
quoted
Hi,

On Tue, Jul 13, 2021 at 8:20 PM Wang ShaoBo
[off-list ref] wrote:
quoted
use-after-free error in lock_sock_nested() is reported:

[  179.140137][ T3731]
=====================================================
[  179.142675][ T3731] BUG: KMSAN: use-after-free in
lock_sock_nested+0x280/0x2c0
[  179.145494][ T3731] CPU: 4 PID: 3731 Comm: kworker/4:2 Not
tainted 5.12.0-rc6+ #54
[  179.148432][ T3731] Hardware name: QEMU Standard PC (i440FX +
PIIX, 1996), BIOS 1.13.0-1ubuntu1.1 04/01/2014
[  179.151806][ T3731] Workqueue: events l2cap_chan_timeout
[  179.152730][ T3731] Call Trace:
[  179.153301][ T3731]  dump_stack+0x24c/0x2e0
[  179.154063][ T3731]  kmsan_report+0xfb/0x1e0
[  179.154855][ T3731]  __msan_warning+0x5c/0xa0
[  179.155579][ T3731]  lock_sock_nested+0x280/0x2c0
[  179.156436][ T3731]  ? kmsan_get_metadata+0x116/0x180
[  179.157257][ T3731]  l2cap_sock_teardown_cb+0xb8/0x890
[  179.158154][ T3731]  ? __msan_metadata_ptr_for_load_8+0x10/0x20
[  179.159141][ T3731]  ? kmsan_get_metadata+0x116/0x180
[  179.159994][ T3731]  ? kmsan_get_shadow_origin_ptr+0x84/0xb0
[  179.160959][ T3731]  ? l2cap_sock_recv_cb+0x420/0x420
[  179.161834][ T3731]  l2cap_chan_del+0x3e1/0x1d50
[  179.162608][ T3731]  ? kmsan_get_metadata+0x116/0x180
[  179.163435][ T3731]  ? kmsan_get_shadow_origin_ptr+0x84/0xb0
[  179.164406][ T3731]  l2cap_chan_close+0xeea/0x1050
[  179.165189][ T3731]  ? kmsan_internal_unpoison_shadow+0x42/0x70
[  179.166180][ T3731]  l2cap_chan_timeout+0x1da/0x590
[  179.167066][ T3731]  ? __msan_metadata_ptr_for_load_8+0x10/0x20
[  179.168023][ T3731]  ? l2cap_chan_create+0x560/0x560
[  179.168818][ T3731]  process_one_work+0x121d/0x1ff0
[  179.169598][ T3731]  worker_thread+0x121b/0x2370
[  179.170346][ T3731]  kthread+0x4ef/0x610
[  179.171010][ T3731]  ? process_one_work+0x1ff0/0x1ff0
[  179.171828][ T3731]  ? kthread_blkcg+0x110/0x110
[  179.172587][ T3731]  ret_from_fork+0x1f/0x30
[  179.173348][ T3731]
[  179.173752][ T3731] Uninit was created at:
[  179.174409][ T3731]  kmsan_internal_poison_shadow+0x5c/0xf0
[  179.175373][ T3731]  kmsan_slab_free+0x76/0xc0
[  179.176060][ T3731]  kfree+0x3a5/0x1180
[  179.176664][ T3731]  __sk_destruct+0x8af/0xb80
[  179.177375][ T3731]  __sk_free+0x812/0x8c0
[  179.178032][ T3731]  sk_free+0x97/0x130
[  179.178686][ T3731]  l2cap_sock_release+0x3d5/0x4d0
[  179.179457][ T3731]  sock_close+0x150/0x450
[  179.180117][ T3731]  __fput+0x6bd/0xf00
[  179.180787][ T3731]  ____fput+0x37/0x40
[  179.181481][ T3731]  task_work_run+0x140/0x280
[  179.182219][ T3731]  do_exit+0xe51/0x3e60
[  179.182930][ T3731]  do_group_exit+0x20e/0x450
[  179.183656][ T3731]  get_signal+0x2dfb/0x38f0
[  179.184344][ T3731]  arch_do_signal_or_restart+0xaa/0xe10
[  179.185266][ T3731]  exit_to_user_mode_prepare+0x2d2/0x560
[  179.186136][ T3731]  syscall_exit_to_user_mode+0x35/0x60
[  179.186984][ T3731]  do_syscall_64+0xc5/0x140
[  179.187681][ T3731] entry_SYSCALL_64_after_hwframe+0x44/0xae
[  179.188604][ T3731]
=====================================================

In our case, there are two Thread A and B:

Context: Thread A:              Context: Thread B:

l2cap_chan_timeout()            __se_sys_shutdown()
   l2cap_chan_close()              l2cap_sock_shutdown()
     l2cap_chan_del()                l2cap_chan_close()
       l2cap_sock_teardown_cb() l2cap_sock_teardown_cb()

Once l2cap_sock_teardown_cb() excuted, this sock will be marked as
SOCK_ZAPPED,
and can be treated as killable in l2cap_sock_kill() if sock_orphan()
has
excuted, at this time we close sock through sock_close() which end
to call
l2cap_sock_kill() like Thread C:

Context: Thread C:

sock_close()
   l2cap_sock_release()
     sock_orphan()
     l2cap_sock_kill()  #free sock if refcnt is 1

If C completed, Once A or B reaches l2cap_sock_teardown_cb() again,
use-after-free happened.

We should set chan->data to NULL if sock is freed, for telling teardown
operation is not allowed in l2cap_sock_teardown_cb(), and also we
should
avoid killing an already killed socket in l2cap_sock_close_cb().

Signed-off-by: Wang ShaoBo <redacted>
---
  net/bluetooth/l2cap_sock.c | 14 ++++++++++++--
  1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/net/bluetooth/l2cap_sock.c b/net/bluetooth/l2cap_sock.c
index c99d65ef13b1..ddc6a692b237 100644
--- a/net/bluetooth/l2cap_sock.c
+++ b/net/bluetooth/l2cap_sock.c
@@ -1215,14 +1215,18 @@ static int l2cap_sock_recvmsg(struct socket
*sock, struct msghdr *msg,
   */
  static void l2cap_sock_kill(struct sock *sk)
  {
+       struct l2cap_chan *chan;
+
         if (!sock_flag(sk, SOCK_ZAPPED) || sk->sk_socket)
                 return;

         BT_DBG("sk %p state %s", sk, state_to_string(sk->sk_state));

         /* Kill poor orphan */
-
-       l2cap_chan_put(l2cap_pi(sk)->chan);
+       chan = l2cap_pi(sk)->chan;
+       l2cap_chan_put(chan);
There is a problem here, the above sentence `l2cap_chan_put(chan)`
should put after

following sentence.
quoted
quoted
+       if (refcount_read(&sk->sk_refcnt) == 1)
+               chan->data = NULL;
Instead of checking if it is the last reference here, wouldn't it be
better to reset the chan->data to NULL on l2cap_sock_destruct?
Hi,

In my case it looks OK, this is the diff:
diff --git a/net/bluetooth/l2cap_sock.c b/net/bluetooth/l2cap_sock.c
index f1b1edd0b697..32ef3328ab49 100644
--- a/net/bluetooth/l2cap_sock.c
+++ b/net/bluetooth/l2cap_sock.c
@@ -1500,6 +1500,9 @@ static void l2cap_sock_close_cb(struct
l2cap_chan *chan)
 {
        struct sock *sk = chan->data;

+       if (!sk)
+               return;
+
        l2cap_sock_kill(sk);
 }
@@ -1508,6 +1511,9 @@ static void l2cap_sock_teardown_cb(struct
l2cap_chan *chan, int err)
        struct sock *sk = chan->data;
        struct sock *parent;

+       if (!sk)
+               return;
+
        BT_DBG("chan %p state %s", chan, state_to_string(chan->state));

        /* This callback can be called both for server (BT_LISTEN)
@@ -1700,6 +1706,7 @@ static void l2cap_sock_destruct(struct sock *sk)
        BT_DBG("sk %p", sk);

        if (l2cap_pi(sk)->chan)
+              l2cap_pi(sk)->chan->data = NULL;
                 l2cap_chan_put(l2cap_pi(sk)->chan);
But if it has potential risk if l2cap_sock_destruct() can not be
excuted in time ?

sk_free():

        if (refcount_dec_and_test(&sk->sk_wmem_alloc)) //is possible
this condition false ?

              __sk_free(sk)   -> ... l2cap_sock_destruct()
Dear Luiz,

Not only that, if l2cap_sock_kill() has put 'l2cap_pi(sk)->chan', how
does we avoid re-puting 'l2cap_pi(sk)->chan' if l2cap_sock_destruct()
work postponed? this will cause underflow of chan->refcount; this PATCH
4e1a720d0312 ("Bluetooth: avoid killing an already killed socket") also
may not work in any case because only sock_orphan() has excuted can this
sock be killed, but if sco_sock_release() excute first, for this sock
has been marked as SOCK_DEAD, this sock can never be killed. So should
we think put chan->data = NULL in xx_sock_kill() is a better choice ?
Not sure what do you mean by postponed? Interrupted perhaps? Even in
that case what are trying to prevent is use after free so if the
callback has not run yet that means the sk has not been freed. Anyway
I think we could do it inconditionally in l2cap_sock_kill since we
will be releasing the reference owned by l2cap_pi(sk)->chan->data that
should be reset to NULL immediatelly.
- WangShaoBo

-- 
Luiz Augusto von Dentz

Re: [PATCH] Bluetooth: fix use-after-free error in lock_sock_nested()

From: Wangshaobo (bobo) <hidden>
Date: 2021-07-19 02:09:49

quoted
quoted
Hi,

In my case it looks OK, this is the diff:
diff --git a/net/bluetooth/l2cap_sock.c b/net/bluetooth/l2cap_sock.c
index f1b1edd0b697..32ef3328ab49 100644
--- a/net/bluetooth/l2cap_sock.c
+++ b/net/bluetooth/l2cap_sock.c
@@ -1500,6 +1500,9 @@ static void l2cap_sock_close_cb(struct
l2cap_chan *chan)
  {
         struct sock *sk = chan->data;

+       if (!sk)
+               return;
+
         l2cap_sock_kill(sk);
  }
@@ -1508,6 +1511,9 @@ static void l2cap_sock_teardown_cb(struct
l2cap_chan *chan, int err)
         struct sock *sk = chan->data;
         struct sock *parent;

+       if (!sk)
+               return;
+
         BT_DBG("chan %p state %s", chan, state_to_string(chan->state));

         /* This callback can be called both for server (BT_LISTEN)
@@ -1700,6 +1706,7 @@ static void l2cap_sock_destruct(struct sock *sk)
         BT_DBG("sk %p", sk);

         if (l2cap_pi(sk)->chan)
+              l2cap_pi(sk)->chan->data = NULL;
                  l2cap_chan_put(l2cap_pi(sk)->chan);
But if it has potential risk if l2cap_sock_destruct() can not be
excuted in time ?

sk_free():

         if (refcount_dec_and_test(&sk->sk_wmem_alloc)) //is possible
this condition false ?

               __sk_free(sk)   -> ... l2cap_sock_destruct()
Dear Luiz,

Not only that, if l2cap_sock_kill() has put 'l2cap_pi(sk)->chan', how
does we avoid re-puting 'l2cap_pi(sk)->chan' if l2cap_sock_destruct()
work postponed? this will cause underflow of chan->refcount; this PATCH
4e1a720d0312 ("Bluetooth: avoid killing an already killed socket") also
may not work in any case because only sock_orphan() has excuted can this
sock be killed, but if sco_sock_release() excute first, for this sock
has been marked as SOCK_DEAD, this sock can never be killed. So should
we think put chan->data = NULL in xx_sock_kill() is a better choice ?
Not sure what do you mean by postponed? Interrupted perhaps? Even in
that case what are trying to prevent is use after free so if the
callback has not run yet that means the sk has not been freed. Anyway
I think we could do it inconditionally in l2cap_sock_kill since we
will be releasing the reference owned by l2cap_pi(sk)->chan->data that
should be reset to NULL immediatelly.
Dear  Luiz,

yes, that's right, if sk can be accessed, it also means that chan has 
not been destroyed, thanks very much.

-- Wang ShaoBo
Keyboard shortcuts
hback out one level
jnext message in thread
kprevious message in thread
ldrill in
Escclose help / fold thread tree
?toggle this help