UBSAN: shift-out-of-bounds in drivers/video/fbdev/core/fb_fillrect.h:100:21 (v6.17-rc2)

Greetings!

Getting this UBSAN hit on my PowerMac G4 DP with kernel 6.17-rc2:

[...]
Console: switching to colour frame buffer device 240x67
------------[ cut here ]------------
UBSAN: shift-out-of-bounds in drivers/video/fbdev/core/fb_fillrect.h:100:21
shift exponent 32 is too large for 32-bit type 'unsigned long'
CPU: 1 UID: 0 PID: 542 Comm: (udev-worker) Tainted: G                 N 6.17.0-rc2-PMacG4 #2 PREEMPTLAZY
Tainted: [N]=TEST
Hardware name: PowerMac3,6 7455 0x80010303 PowerMac
Call Trace:
[c20fb270] [c0ac2494] __dump_stack+0x28/0x3c (unreliable)
[c20fb280] [c0ac244c] dump_stack_lvl+0x68/0x88
[c20fb2a0] [c0ac24c0] dump_stack+0x18/0x28
[c20fb2b0] [c06d8298] ubsan_epilogue+0x14/0x50
[c20fb2c0] [c06d7f3c] __ubsan_handle_shift_out_of_bounds+0x224/0x234
[c20fb350] [c07194b4] cfb_fillrect+0x9c4/0x9c8
[c20fb3c0] [c07181d4] bit_clear_margins+0xe8/0x108
[c20fb400] [c0714a0c] fbcon_clear_margins+0xa0/0xd8
[c20fb420] [c0715ce0] fbcon_switch+0x3c0/0x510
[c20fb500] [c0743934] redraw_screen+0x134/0x200
[c20fb530] [c0745ab0] do_bind_con_driver+0x41c/0x458
[c20fb590] [c0745eb0] do_take_over_console+0x18c/0x1e4
[c20fb5c0] [c0713f90] do_fbcon_takeover+0xf8/0x1bc
[c20fb5f0] [c0712bec] fbcon_fb_registered+0x1e8/0x2a8
[c20fb620] [c070e0bc] register_framebuffer+0x22c/0x2d0
[c20fb680] [beb146f4] __drm_fb_helper_initial_config_and_unlock+0x4b0/0x674 [drm_kms_helper]
[c20fb700] [beb14218] drm_fb_helper_initial_config+0x44/0x70 [drm_kms_helper]
[c20fb720] [beb413c4] drm_fbdev_client_hotplug+0x90/0x104 [drm_client_lib]
[c20fb740] [c07ac3c4] drm_client_register+0x90/0xfc
[c20fb770] [beb4114c] drm_fbdev_client_setup+0x110/0x278 [drm_client_lib]
[c20fb790] [beb40278] drm_client_setup+0xc0/0x134 [drm_client_lib]
[c20fb7a0] [bebaa494] radeon_pci_probe+0x220/0x228 [radeon]
[c20fb7c0] [c06eee78] pci_device_probe+0xc4/0x190
[c20fb7f0] [c07c309c] really_probe+0xf4/0x2d8
[c20fb810] [c07c24c8] __driver_probe_device+0xa4/0x114
[c20fb830] [c07c2f0c] driver_probe_device+0x4c/0xe8
[c20fb850] [c07c26b0] __driver_attach+0xcc/0x128
[c20fb870] [c07bfc38] bus_for_each_dev+0xa4/0xe8
[c20fb8a0] [c07c25d4] driver_attach+0x24/0x34
[c20fb8b0] [c07c0380] bus_add_driver+0x20c/0x2e0
[c20fb8e0] [c07c3d4c] driver_register+0x8c/0x154
[c20fb900] [c06eeaa8] __pci_register_driver+0x74/0x88
[c20fb910] [beba60bc] init_module+0x8c/0xfd0 [radeon]
[c20fb920] [c0007958] do_one_initcall+0xf0/0x2d8
[c20fbc10] [c00fdf08] do_init_module+0x90/0x33c
[c20fbc30] [c00fd0cc] load_module+0x1428/0x14bc
[c20fbc80] [c00fafac] sys_finit_module+0x250/0x350
[c20fbd40] [c0012d60] system_call_exception+0xe0/0x204
[c20fbf30] [c00181ac] ret_from_syscall+0x0/0x2c
---- interrupt: c00 at 0x43fc94
NIP:  0043fc94 LR: 0054c254 CTR: 00453790
REGS: c20fbf40 TRAP: 0c00   Tainted: G                 N (6.17.0-rc2-PMacG4)
MSR:  0000d032 <EE,PR,ME,IR,DR,RI>  CR: 2822242c  XER: 20000000

GPR00: 00000161 af93bb50 a7ae5880 00000023 005583e8 00000000 af93bb25 0000007f
GPR08: 00000000 00000000 0000002f 0a565c56 4422842c 00a9f71c 2822442c 00000000
GPR16: 00020000 0aba9500 00000000 00000000 010b1dc0 00000000 010b86c0 af93bd3c
GPR24: 010b1dc0 00000000 00020000 010a6400 005583e8 00000000 00577ad0 010b1dc0
NIP [0043fc94] 0x43fc94
LR [0054c254] 0x54c254
---- interrupt: c00
---[ end trace ]---
ADM1030 fan controller [@2c]
DS1775 digital thermometer [@49]
radeon 0000:00:10.0: [drm] fb0: radeondrmfb frame buffer device
[...]

I guess this would be a problem on other 32bit arches too?

If needed I can attach full dmesg, kernel .config lspci output.

Regards,
Erhard

Greetings!

Getting this UBSAN hit on my PowerMac G4 DP with kernel 6.17-rc2:

[...]
Console: switching to colour frame buffer device 240x67
------------[ cut here ]------------
UBSAN: shift-out-of-bounds in drivers/video/fbdev/core/fb_fillrect.h:100:21
shift exponent 32 is too large for 32-bit type 'unsigned long'

Thanks for reporting!

I guess this would be a problem on other 32bit arches too?

It's only on 32 bit big endian. I don't have UBSAN for MIPS on my setup
so haven't noticed it.

#ifndef __LITTLE_ENDIAN
        pattern <<= (BITS_PER_LONG % bpp);
        pattern |= pattern >> bpp;          <-
#endif

In the 32 BPP case the result is identical in both the no shift and zero
result implementations.

I've patched it by skipping this realignment as it's only needed if the
BPP is smaller than the word length.

-- 
						    -soci-

It's only on 32 bit big endian. I don't have UBSAN for MIPS on my setup
so haven't noticed it.

#ifndef __LITTLE_ENDIAN
         pattern <<= (BITS_PER_LONG % bpp);
         pattern |= pattern >> bpp;          <-
#endif

In the 32 BPP case the result is identical in both the no shift and zero
result implementations.

I've patched it by skipping this realignment as it's only needed if the
BPP is smaller than the word length.

Thanks for looking into this!

Applied your patch from https://lore.kernel.org/linux-fbdev/20250821024248.7458-1-soci@c64.rulez.org/T/#u which fixes the USBAN hit for me.

Greetings,
Erhard