[PATCH] task_work: kasan: record task_work_add() call stack

Why record task_work_add() call stack?
Syzbot reports many use-after-free issues for task_work, see [1].
After see the free stack and the current auxiliary stack, we think
they are useless, we don't know where register the work, this work
may be the free call stack, so that we miss the root cause and
don't solve the use-after-free.

Add task_work_add() call stack into KASAN auxiliary stack in
order to improve KASAN report. It is useful for programmers
to solve use-after-free issues.

[1]: https://groups.google.com/g/syzkaller-bugs/search?q=kasan%20use-after-free%20task_work_run

Signed-off-by: Walter Wu <redacted>
Suggested-by: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Konovalov <redacted>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Matthias Brugger <matthias.bgg@gmail.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Oleg Nesterov <oleg@redhat.com>
---
 kernel/task_work.c | 3 +++
 mm/kasan/kasan.h   | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/kernel/task_work.c b/kernel/task_work.c
index 9cde961875c0..f255294377da 100644
--- a/kernel/task_work.c
+++ b/kernel/task_work.c

@@ -55,6 +55,9 @@ int task_work_add(struct task_struct *task, struct callback_head *work,
 		break;
 	}
 
+	/* record the work call stack in order to print it in KASAN reports */
+	kasan_record_aux_stack(work);
+
 	return 0;
 }
 

diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index 3436c6bf7c0c..d300fe9415bd 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h

@@ -146,7 +146,7 @@ struct kasan_alloc_meta {
 	struct kasan_track alloc_track;
 #ifdef CONFIG_KASAN_GENERIC
 	/*
-	 * call_rcu() call stack is stored into struct kasan_alloc_meta.
+	 * Auxiliary stack is stored into struct kasan_alloc_meta.
 	 * The free stack is stored into struct kasan_free_meta.
 	 */
 	depot_stack_handle_t aux_stack[2];

-- 
2.18.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

On Mon, Mar 15, 2021 at 3:00 AM Walter Wu [off-list ref] wrote:

I think this call should be done _before_ we actually queue the work,
because this function may operate on non-current task.
Consider, we queue the work, the other task already executes it and
triggers use-after-free, now only now we record the stack.
Moreover, I think we can trigger use-after-free here ourselves while
recording the aux stack. We queued the work, and the work can cause
own free, so it's not necessary live by now.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

On Mon, 2021-03-15 at 07:58 +0100, 'Dmitry Vyukov' via kasan-dev wrote:

On Mon, Mar 15, 2021 at 3:00 AM Walter Wu [off-list ref] wrote:

quoted

Why record task_work_add() call stack?
Syzbot reports many use-after-free issues for task_work, see [1].
After see the free stack and the current auxiliary stack, we think
they are useless, we don't know where register the work, this work
may be the free call stack, so that we miss the root cause and
don't solve the use-after-free.

Add task_work_add() call stack into KASAN auxiliary stack in
order to improve KASAN report. It is useful for programmers
to solve use-after-free issues.

[1]: https://groups.google.com/g/syzkaller-bugs/search?q=kasan%20use-after-free%20task_work_run

Signed-off-by: Walter Wu <redacted>
Suggested-by: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Konovalov <redacted>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Matthias Brugger <matthias.bgg@gmail.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Oleg Nesterov <oleg@redhat.com>
---
 kernel/task_work.c | 3 +++
 mm/kasan/kasan.h   | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/kernel/task_work.c b/kernel/task_work.c
index 9cde961875c0..f255294377da 100644
--- a/kernel/task_work.c
+++ b/kernel/task_work.c

@@ -55,6 +55,9 @@ int task_work_add(struct task_struct *task, struct callback_head *work,
                break;
        }

+       /* record the work call stack in order to print it in KASAN reports */
+       kasan_record_aux_stack(work);

I think this call should be done _before_ we actually queue the work,
because this function may operate on non-current task.
Consider, we queue the work, the other task already executes it and
triggers use-after-free, now only now we record the stack.

agree, what do you think below change?

--- a/kernel/task_work.c
+++ b/kernel/task_work.c

@@ -34,6 +34,9 @@ int task_work_add(struct task_struct *task, struct

callback_head *work,
 {
    struct callback_head *head;

+   /* record the work call stack in order to print it in KASAN reports
*/
+   kasan_record_aux_stack(work);
+
    do {
        head = READ_ONCE(task->task_works);
        if (unlikely(head == &work_exited))

@@ -55,9 +58,6 @@ int task_work_add(struct task_struct *task, struct

callback_head *work,
        break;
    }

-   /* record the work call stack in order to print it in KASAN reports
*/
-   kasan_record_aux_stack(work);
-
    return 0;
 }

Moreover, I think we can trigger use-after-free here ourselves while
recording the aux stack. We queued the work, and the work can cause
own free, so it's not necessary live by now.

Sorry, I don't fully know your meaning, do you mean we should add an
abort when detect use-after-free?

quoted

        return 0;
 }

diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index 3436c6bf7c0c..d300fe9415bd 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h

@@ -146,7 +146,7 @@ struct kasan_alloc_meta {
        struct kasan_track alloc_track;
 #ifdef CONFIG_KASAN_GENERIC
        /*
-        * call_rcu() call stack is stored into struct kasan_alloc_meta.
+        * Auxiliary stack is stored into struct kasan_alloc_meta.
         * The free stack is stored into struct kasan_free_meta.
         */
        depot_stack_handle_t aux_stack[2];
--

2.18.0

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

On Mon, Mar 15, 2021 at 10:38 AM Walter Wu [off-list ref] wrote:

This looks good to me.

I meant that where we had the kasan_record_aux_stack(work) call in the
first version of the patch, work can be already freed. We must not
access work after queueing it.

quoted

quoted

        return 0;
 }

diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index 3436c6bf7c0c..d300fe9415bd 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h

@@ -146,7 +146,7 @@ struct kasan_alloc_meta {
        struct kasan_track alloc_track;
 #ifdef CONFIG_KASAN_GENERIC
        /*
-        * call_rcu() call stack is stored into struct kasan_alloc_meta.
+        * Auxiliary stack is stored into struct kasan_alloc_meta.
         * The free stack is stored into struct kasan_free_meta.
         */
        depot_stack_handle_t aux_stack[2];
--

2.18.0

--
You received this message because you are subscribed to the Google Groups "kasan-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/1615801102.24887.4.camel%40mtksdccf07.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

On Mon, 2021-03-15 at 11:03 +0100, 'Dmitry Vyukov' via kasan-dev wrote:

On Mon, Mar 15, 2021 at 10:38 AM Walter Wu [off-list ref] wrote:

quoted

On Mon, 2021-03-15 at 07:58 +0100, 'Dmitry Vyukov' via kasan-dev wrote:

quoted

On Mon, Mar 15, 2021 at 3:00 AM Walter Wu [off-list ref] wrote:

quoted

Why record task_work_add() call stack?
Syzbot reports many use-after-free issues for task_work, see [1].
After see the free stack and the current auxiliary stack, we think
they are useless, we don't know where register the work, this work
may be the free call stack, so that we miss the root cause and
don't solve the use-after-free.

Add task_work_add() call stack into KASAN auxiliary stack in
order to improve KASAN report. It is useful for programmers
to solve use-after-free issues.

[1]: https://groups.google.com/g/syzkaller-bugs/search?q=kasan%20use-after-free%20task_work_run

Signed-off-by: Walter Wu <redacted>
Suggested-by: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Andrey Konovalov <redacted>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Matthias Brugger <matthias.bgg@gmail.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Oleg Nesterov <oleg@redhat.com>
---
 kernel/task_work.c | 3 +++
 mm/kasan/kasan.h   | 2 +-
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/kernel/task_work.c b/kernel/task_work.c
index 9cde961875c0..f255294377da 100644
--- a/kernel/task_work.c
+++ b/kernel/task_work.c

@@ -55,6 +55,9 @@ int task_work_add(struct task_struct *task, struct callback_head *work,
                break;
        }

+       /* record the work call stack in order to print it in KASAN reports */
+       kasan_record_aux_stack(work);

I think this call should be done _before_ we actually queue the work,
because this function may operate on non-current task.
Consider, we queue the work, the other task already executes it and
triggers use-after-free, now only now we record the stack.

agree, what do you think below change?

--- a/kernel/task_work.c
+++ b/kernel/task_work.c

@@ -34,6 +34,9 @@ int task_work_add(struct task_struct *task, struct

callback_head *work,
 {
    struct callback_head *head;

+   /* record the work call stack in order to print it in KASAN reports
*/
+   kasan_record_aux_stack(work);
+

This looks good to me.

quoted

    do {
        head = READ_ONCE(task->task_works);
        if (unlikely(head == &work_exited))

@@ -55,9 +58,6 @@ int task_work_add(struct task_struct *task, struct

callback_head *work,
        break;
    }

-   /* record the work call stack in order to print it in KASAN reports
*/
-   kasan_record_aux_stack(work);
-
    return 0;
 }

quoted

Moreover, I think we can trigger use-after-free here ourselves while
recording the aux stack. We queued the work, and the work can cause
own free, so it's not necessary live by now.

Sorry, I don't fully know your meaning, do you mean we should add an
abort when detect use-after-free?

I meant that where we had the kasan_record_aux_stack(work) call in the
first version of the patch, work can be already freed. We must not
access work after queueing it.

Got it. Now I must treat urgent issue, I will send v2 patch tomorrow.

Thanks for your review.

quoted

quoted

quoted

        return 0;
 }

diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index 3436c6bf7c0c..d300fe9415bd 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h

@@ -146,7 +146,7 @@ struct kasan_alloc_meta {
        struct kasan_track alloc_track;
 #ifdef CONFIG_KASAN_GENERIC
        /*
-        * call_rcu() call stack is stored into struct kasan_alloc_meta.
+        * Auxiliary stack is stored into struct kasan_alloc_meta.
         * The free stack is stored into struct kasan_free_meta.
         */
        depot_stack_handle_t aux_stack[2];
--

2.18.0

--
You received this message because you are subscribed to the Google Groups "kasan-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/1615801102.24887.4.camel%40mtksdccf07.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel