Re: bpf_get_branch_snapshot on qemu-kvm

Hi Like,

On Sep 29, 2021, at 5:05 AM, Like Xu [off-list ref] wrote:

On 29/9/2021 3:35 pm, Peter Zijlstra wrote:

quoted

On Wed, Sep 29, 2021 at 12:04:21AM +0000, Song Liu wrote:

quoted

Hi Peter,

We have see the warning below while testing the new bpf_get_branch_snapshot
helper, on a QEMU vm (/usr/bin/qemu-system-x86_64 -enable-kvm -cpu host ...).
This issue doesn't happen on bare metal systems (no QEMU).

We didn't cover this case, as LBR didn't really work in QEMU. But it seems to
work after I upgrade the host kernel to 5.12.

The guest LBR is enabled since the v5.12.

quoted

quoted

At the moment, we don't have much idea on how to debug and fix the issue. Could
you please share your thoughts on this?

Well, that's virt, afaik stuff not working is like a feature there or
something, who knows. I've Cc'ed Like Xu who might have clue since he
did the patches.
Virt just ain't worth the pain if you ask me.

Just cc me for any vPMU/x86 stuff.

quoted

quoted

Thanks in advance!

Song




============================== 8< ============================

[  139.494159] unchecked MSR access error: WRMSR to 0x3f1 (tried to write 0x0000000000000000) at rIP: 0xffffffff81011a8b (intel_pmu_snapshot_branch_stack+0x3b/0xd0)

Uh, it uses a PEBS counter to sample or count, which is not yet upstream but should be soon.

Song, can you try to fix bpf_get_branch_snapshot on a normal PMC counter,
or where is the src for bpf_get_branch_snapshot? I am more than happy to help.

bpf_get_branch_snapshot is available in the bpf-next tree:

https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/

To repro the issue:

1. build the kernel and boot qemu with it;
2. build tools/testing/selftests/bpf
     cd tools/testing/selftests/bpf ; make -j
3. copy test_progs and bpf_testmod.ko from tools/testing/selftests/bpf
   to the qemu vm;
4. run the test as 
     ./test_progs -t snapshot -v

This should trigger "unchecked MSR access error" in dmesg. Please 
let me know if this doesn't work. 

Thanks,
Song

quoted

quoted

[  139.495587] Call Trace:
[  139.495845]  bpf_get_branch_snapshot+0x17/0x40
[  139.496285]  bpf_prog_35810402cd1d294c_test1+0x33/0xe6c
[  139.496791]  bpf_trampoline_10737534536_0+0x4c/0x1000
[  139.497274]  bpf_testmod_loop_test+0x5/0x20 [bpf_testmod]
[  139.497799]  bpf_testmod_test_read+0x71/0x1f0 [bpf_testmod]
[  139.498332]  ? bpf_testmod_loop_test+0x20/0x20 [bpf_testmod]
[  139.498878]  ? sysfs_kf_bin_read+0xbe/0x110
[  139.499284]  ? bpf_testmod_loop_test+0x20/0x20 [bpf_testmod]
[  139.499829]  kernfs_fop_read_iter+0x1ac/0x2c0
[  139.500245]  ? kernfs_create_link+0x110/0x110
[  139.500667]  new_sync_read+0x24b/0x360
[  139.501037]  ? __x64_sys_llseek+0x1e0/0x1e0
[  139.501444]  ? rcu_read_lock_held_common+0x1a/0x50
[  139.501942]  ? rcu_read_lock_held_common+0x1a/0x50
[  139.502404]  ? rcu_read_lock_sched_held+0x5f/0xd0
[  139.502865]  ? rcu_read_lock_bh_held+0xb0/0xb0
[  139.503294]  ? security_file_permission+0xe7/0x2c0
[  139.503758]  vfs_read+0x1a4/0x2a0
[  139.504091]  ksys_read+0xc0/0x160
[  139.504413]  ? vfs_write+0x510/0x510
[  139.504756]  ? ktime_get_coarse_real_ts64+0xe4/0xf0
[  139.505234]  do_syscall_64+0x3a/0x80
[  139.505581]  entry_SYSCALL_64_after_hwframe+0x44/0xae
[  139.506066] RIP: 0033:0x7fb8a05728b2
[  139.506413] Code: 97 20 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b6 0f 1f 80 00 00 00 00 f3 0f 1e fa 8b 05 96 db 20 00 85 c0 75 12 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 56 c3 0f 1f 44 00 00 41 54 49 89 d4 55 48 89
[  139.508164] RSP: 002b:00007ffe66315a28 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[  139.508870] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007fb8a05728b2
[  139.509545] RDX: 0000000000000064 RSI: 0000000000000000 RDI: 0000000000000010
[  139.510225] RBP: 00007ffe66315a60 R08: 0000000000000000 R09: 00007ffe66315907
[  139.510897] R10: 0000000000000000 R11: 0000000000000246 R12: 000000000040c8b0
[  139.511570] R13: 00007ffe66315cc0 R14: 0000000000000000 R15: 0000000000000000