From: Pavel Skripkin <hidden> Date: 2021-10-22 18:33:54
On 10/22/21 02:19, syzbot wrote:
Hello,
syzbot found the following issue on:
HEAD commit: 2f111a6fd5b5 Merge tag 'ceph-for-5.15-rc7' of git://github..
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=115750acb00000
kernel config: https://syzkaller.appspot.com/x/.config?x=d95853dad8472c91
dashboard link: https://syzkaller.appspot.com/bug?extid=28b0702ada0bf7381f58
compiler: Debian clang version 11.0.1-2, GNU ld (GNU Binutils for Debian) 2.35.2
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1026ef2cb00000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=15c9c162b00000
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+28b0702ada0bf7381f58@syzkaller.appspotmail.com
RBP: 00007ffef262e230 R08: 0000000000000002 R09: 00007fddc8003531
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000004
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
------------[ cut here ]------------
ODEBUG: assert_init not available (active state 0) object type: timer_list hint: 0x0
WARNING: CPU: 0 PID: 6517 at lib/debugobjects.c:508 debug_print_object lib/debugobjects.c:505 [inline]
WARNING: CPU: 0 PID: 6517 at lib/debugobjects.c:508 debug_object_assert_init+0x1fa/0x250 lib/debugobjects.c:895
Modules linked in:
CPU: 0 PID: 6517 Comm: syz-executor011 Not tainted 5.15.0-rc6-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
RIP: 0010:debug_print_object lib/debugobjects.c:505 [inline]
RIP: 0010:debug_object_assert_init+0x1fa/0x250 lib/debugobjects.c:895
Code: e8 4b 15 b8 fd 4c 8b 45 00 48 c7 c7 a0 31 b4 8a 48 c7 c6 00 2e b4 8a 48 c7 c2 e0 33 b4 8a 31 c9 49 89 d9 31 c0 e8 b6 c6 36 fd <0f> 0b ff 05 3a 5c c5 09 48 83 c5 38 48 89 e8 48 c1 e8 03 42 80 3c
RSP: 0018:ffffc90002c7e698 EFLAGS: 00010046
RAX: cffa606352c78700 RBX: 0000000000000000 RCX: ffff888076ce9c80
RDX: 0000000000000000 RSI: 0000000080000000 RDI: 0000000000000000
RBP: ffffffff8a512d00 R08: ffffffff81693402 R09: ffffed1017383f2c
R10: ffffed1017383f2c R11: 0000000000000000 R12: dffffc0000000000
R13: ffff88801bcd1720 R14: 0000000000000002 R15: ffffffff90ba5a20
FS: 0000555557087300(0000) GS:ffff8880b9c00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5473f3c000 CR3: 0000000070ca6000 CR4: 00000000003506f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
debug_timer_assert_init kernel/time/timer.c:739 [inline]
debug_assert_init kernel/time/timer.c:784 [inline]
del_timer+0xa5/0x3d0 kernel/time/timer.c:1204
try_to_grab_pending+0x151/0xbb0 kernel/workqueue.c:1270
__cancel_work_timer+0x14c/0x710 kernel/workqueue.c:3129
batadv_nc_mesh_free+0x4a/0xf0 net/batman-adv/network-coding.c:1869
batadv_mesh_free+0x6f/0x140 net/batman-adv/main.c:245
batadv_mesh_init+0x4e5/0x550 net/batman-adv/main.c:226
Looks like cancel_delayed_work_sync() is called before
INIT_DELAYED_WORK(), so calltrace looks like
batadv_mesh_init()
batadv_originator_init() <- injected allocation failure
batadv_mesh_free()
batadv_nc_mesh_free()
cancel_delayed_work_sync()
Quick fix can be moving INIT_DELAYED_WORK() from batadv_nc_init() to
batadv_mesh_init(), since there is complex dependencies between each
mech part, if I understood comments correctly
Just for thoughts and syzbot testing
#syz test
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
With regards,
Pavel Skripkin
From: Pavel Skripkin <hidden> Date: 2021-10-22 20:57:22
On 10/22/21 23:20, syzbot wrote:
Hello,
syzbot has tested the proposed patch but the reproducer is still triggering an issue:
general protection fault in batadv_nc_purge_paths
Oh, ok. Next clean up call in batadv_nc_mesh_free() caused GPF, since
fields are not initialized. Let's try to clean up one by one and do not
break dependencies.
Quite ugly one, but idea is correct, I guess
Also, make each *_init() call clean up all allocated stuff to not call
corresponding *_free() on error handling path, since it introduces
problems, as syzbot reported
With regards,
Pavel Skripkin
From: Pavel Skripkin <hidden> Date: 2021-10-22 20:58:21
On 10/22/21 23:57, Pavel Skripkin wrote:
On 10/22/21 23:20, syzbot wrote:
quoted
Hello,
syzbot has tested the proposed patch but the reproducer is still triggering an issue:
general protection fault in batadv_nc_purge_paths
Oh, ok. Next clean up call in batadv_nc_mesh_free() caused GPF, since
fields are not initialized. Let's try to clean up one by one and do not
break dependencies.
Quite ugly one, but idea is correct, I guess
Also, make each *_init() call clean up all allocated stuff to not call
corresponding *_free() on error handling path, since it introduces
problems, as syzbot reported
Whooops.... Forgot to ask syzbot to test the patch
#syz test
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master
With regards,
Pavel Skripkin
From: Sven Eckelmann <sven@narfation.org> Date: 2021-10-23 07:41:19
On Friday, 22 October 2021 22:58:15 CEST Pavel Skripkin wrote:
[...]
quoted
Oh, ok. Next clean up call in batadv_nc_mesh_free() caused GPF, since
fields are not initialized. Let's try to clean up one by one and do not
break dependencies.
Quite ugly one, but idea is correct, I guess
Also, make each *_init() call clean up all allocated stuff to not call
corresponding *_free() on error handling path, since it introduces
problems, as syzbot reported
Thanks for the patch + syzbot interactions. I just wanted to implement a
change - which would most likely have ended up the same way. Can you please
send it to netdev and Cc b.a.t.m.a.n@lists.open-mesh.org? We don't have
anything else to submit at the moment for netdev and this patch can be applied
by netdev directly. I will add my Acked-by in this process.
Not sure about the Fixes. It is definitely wrong in the initial commit.... but
it got only really problematic when other features got introduced. I would
still say that the initial one should be mentioned.
Fixes: c6c8fea29769 ("net: Add batman-adv meshing protocol")
@Linus, @Marek, @Antonio: Please check whether it is ok to move the
batadv_v_mesh_init after batadv_tt_init + batadv_originator_init.
batadv_v_mesh_init is basically there to initialize:
* bat_priv->bat_v.ogm_buff(|_len|_mutex)
* bat_priv->bat_v.ogm_seqno
* bat_priv->bat_v.ogm_wq
batadv_originator_init is there to initialize the
* bat_priv->orig_hash
* bat_priv->orig_work (batadv_purge_orig) + queue it up
batadv_tt_init is a lot more complex but should in theory not interact with
ogm specific algo ops.
I wouldn't know why there could be a problem but I would leave it to the
experts.
Kind regards,
Sven
From: Pavel Skripkin <hidden> Date: 2021-10-24 13:14:12
Syzbot reported ODEBUG warning in batadv_nc_mesh_free(). The problem was
in wrong error handling in batadv_mesh_init().
Before this patch batadv_mesh_init() was calling batadv_mesh_free() in case
of any batadv_*_init() calls failure. This approach may work well, when
there is some kind of indicator, which can tell which parts of batadv are
initialized; but there isn't any.
All written above lead to cleaning up uninitialized fields. Even if we hide
ODEBUG warning by initializing bat_priv->nc.work, syzbot was able to hit
GPF in batadv_nc_purge_paths(), because hash pointer in still NULL. [1]
To fix these bugs we can unwind batadv_*_init() calls one by one.
It is good approach for 2 reasons: 1) It fixes bugs on error handling
path 2) It improves the performance, since we won't call unneeded
batadv_*_free() functions.
So, this patch makes all batadv_*_init() clean up all allocated memory
before returning with an error to no call correspoing batadv_*_free()
and open-codes batadv_mesh_free() with proper order to avoid touching
uninitialized fields.
Link: https://lore.kernel.org/netdev/000000000000c87fbd05cef6bcb0@google.com/ [1]
Reported-and-tested-by: syzbot+28b0702ada0bf7381f58@syzkaller.appspotmail.com
Fixes: c6c8fea29769 ("net: Add batman-adv meshing protocol")
Signed-off-by: Pavel Skripkin <redacted>
---
net/batman-adv/bridge_loop_avoidance.c | 8 +++-
net/batman-adv/main.c | 56 ++++++++++++++++++--------
net/batman-adv/network-coding.c | 4 +-
net/batman-adv/translation-table.c | 4 +-
4 files changed, 52 insertions(+), 20 deletions(-)
From: Sven Eckelmann <sven@narfation.org> Date: 2021-10-24 14:58:49
On Sunday, 24 October 2021 15:13:56 CEST Pavel Skripkin wrote:
Syzbot reported ODEBUG warning in batadv_nc_mesh_free(). The problem was
in wrong error handling in batadv_mesh_init().
Before this patch batadv_mesh_init() was calling batadv_mesh_free() in case
of any batadv_*_init() calls failure. This approach may work well, when
there is some kind of indicator, which can tell which parts of batadv are
initialized; but there isn't any.
All written above lead to cleaning up uninitialized fields. Even if we hide
ODEBUG warning by initializing bat_priv->nc.work, syzbot was able to hit
GPF in batadv_nc_purge_paths(), because hash pointer in still NULL. [1]
To fix these bugs we can unwind batadv_*_init() calls one by one.
It is good approach for 2 reasons: 1) It fixes bugs on error handling
path 2) It improves the performance, since we won't call unneeded
batadv_*_free() functions.
So, this patch makes all batadv_*_init() clean up all allocated memory
before returning with an error to no call correspoing batadv_*_free()
and open-codes batadv_mesh_free() with proper order to avoid touching
uninitialized fields.
Link: https://lore.kernel.org/netdev/000000000000c87fbd05cef6bcb0@google.com/ [1]
Reported-and-tested-by: syzbot+28b0702ada0bf7381f58@syzkaller.appspotmail.com
Fixes: c6c8fea29769 ("net: Add batman-adv meshing protocol")
Signed-off-by: Pavel Skripkin <redacted>
Acked-by: Sven Eckelmann <sven@narfation.org>
Kind regards,
Sven
From: Jakub Kicinski <kuba@kernel.org> Date: 2021-10-26 00:49:54
On Sun, 24 Oct 2021 16:58:30 +0200 Sven Eckelmann wrote:
On Sunday, 24 October 2021 15:13:56 CEST Pavel Skripkin wrote:
quoted
Syzbot reported ODEBUG warning in batadv_nc_mesh_free(). The problem was
in wrong error handling in batadv_mesh_init().
Before this patch batadv_mesh_init() was calling batadv_mesh_free() in case
of any batadv_*_init() calls failure. This approach may work well, when
there is some kind of indicator, which can tell which parts of batadv are
initialized; but there isn't any.
All written above lead to cleaning up uninitialized fields. Even if we hide
ODEBUG warning by initializing bat_priv->nc.work, syzbot was able to hit
GPF in batadv_nc_purge_paths(), because hash pointer in still NULL. [1]
To fix these bugs we can unwind batadv_*_init() calls one by one.
It is good approach for 2 reasons: 1) It fixes bugs on error handling
path 2) It improves the performance, since we won't call unneeded
batadv_*_free() functions.
So, this patch makes all batadv_*_init() clean up all allocated memory
before returning with an error to no call correspoing batadv_*_free()
and open-codes batadv_mesh_free() with proper order to avoid touching
uninitialized fields.
Link: https://lore.kernel.org/netdev/000000000000c87fbd05cef6bcb0@google.com/ [1]
Reported-and-tested-by: syzbot+28b0702ada0bf7381f58@syzkaller.appspotmail.com
Fixes: c6c8fea29769 ("net: Add batman-adv meshing protocol")
Signed-off-by: Pavel Skripkin <redacted>
Acked-by: Sven Eckelmann <sven@narfation.org>
FWIW I'm marking this as "Awaiting upstream" in netdev patchwork,
please LMK if you prefer for it to be applied directly.
Hello:
This patch was applied to netdev/net.git (master)
by David S. Miller [off-list ref]:
On Sun, 24 Oct 2021 16:13:56 +0300 you wrote:
Syzbot reported ODEBUG warning in batadv_nc_mesh_free(). The problem was
in wrong error handling in batadv_mesh_init().
Before this patch batadv_mesh_init() was calling batadv_mesh_free() in case
of any batadv_*_init() calls failure. This approach may work well, when
there is some kind of indicator, which can tell which parts of batadv are
initialized; but there isn't any.
[...]