Re: [syzbot] kernel panic: corrupted stack end in openat

On Wed, Mar 17, 2021 at 9:32 AM Arnd Bergmann [off-list ref] wrote:

quoted

quoted

quoted

quoted

[off-list ref] wrote:

quoted

On Tue, Mar 16, 2021 at 04:44:45PM +0100, Arnd Bergmann wrote:

quoted

On Tue, Mar 16, 2021 at 11:17 AM Dmitry Vyukov [off-list ref] wrote:

quoted

The compiler is gcc version 10.2.1 20210110 (Debian 10.2.1-6)

Ok, building with Ubuntu 10.2.1-1ubuntu1 20201207 locally, that's
the closest I have installed, and I think the Debian and Ubuntu versions
are generally quite close in case of gcc since they are maintained by
the same packagers.

... which shouldn't be a problem - that's just over 1/4 of the stack
space. Could it be the syzbot's gcc is doing something weird and
inflating the stack frames?

It's possible, I think that's really unlikely given that it's just Debian's
gcc, which is as close to mainline as the version I was using.

Uwe's DEBUG_STACKOVERFLOW patch from a while ago might
help if this was the problem though:
https://lore.kernel.org/linux-arm-kernel/20200108082913.29710-1-u.kleine-koenig@pengutronix.de/ (local)

My best guess is something going wrong in the interrupt
that triggered the preempt_schedule() which ended up calling
task_stack_end_corrupted() in schedule_debug(), as you suggested
earlier.

FWIW I see slightly larger frames with the config:

073ab64 <ima_calc_field_array_hash_tfm>:
8073ab64:       e1a0c00d        mov     ip, sp
8073ab68:       e92ddff0        push    {r4, r5, r6, r7, r8, r9, sl,
fp, ip, lr, pc}
8073ab6c:       e24cb004        sub     fp, ip, #4
8073ab70:       e24ddfa7        sub     sp, sp, #668    ; 0x29c

Yes, this is the one that the compiler complained about when warning
for stack over 600 bytes. It's not called in this call chain though.

quoted

page_alloc can also do reclaim, I had the impression that reclaim can
be quite heavy-weight in all respects.

Yes, that is another possibility. What writable file systems or swap
do you normally have mounted that it could be writing to, and on
what storage device?

The root fs is ext4 on virtio-blk.

There are also several dozens of shrinkers that can be called during reclaim:
https://elixir.bootlin.com/linux/latest/C/ident/unregister_shrinker

Right, unfortunately I don't see a smoking gun there either, unless you are
also using NFS or devicemapper.

Implementing VMAP_STACK as you suggested earlier is probably the
best way to figure out if there is an actual overrun of the stack.
Alternatively, adding support for GCC_PLUGIN_STACKLEAK might
also help find out if we ever get close to the limit. This is probably
less work, but it might not actually help in this case.

VMAP_STACK is quite intrusive as far as I understand. For KASAN I
considered a simpler option: have a debug config that allocates an
extra page after the stack and mprotect's it. It wastes a physical
page per task (fine for a debug config), but I would assume should be
radically simpler to implement. In the end somebody implemented proper
VMAP_STACK support for KASAN, but I still think it may be a reasonable
compromise between time investment and value.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel