On Wed, Aug 10, 2022 at 02:42:40PM -0400, Chris Murphy wrote:

On Wed, Aug 10, 2022, at 2:33 PM, Chris Murphy wrote:

quoted

On Wed, Aug 10, 2022, at 1:48 PM, Josef Bacik wrote:

quoted

To help narrow this down can you disable any IO controller you've got enabled
and see if you can reproduce?  If you can sysrq+w is super helpful as it'll
point us in the next direction to look.  Thanks,

I'm not following, sorry. I can boot with 
systemd.unified_cgroup_hierarchy=0 to make sure it's all off, but we're 
not using an IO cgroup controllers specifically as far as I'm aware.

OK yeah that won't work because the workload requires cgroup2 or it won't run.

Oh no I don't want cgroups completley off, just disable the io controller, so
figure out which cgroup your thing is being run in, and then

echo "-io" > <parent dir>/cgroup.subtree_control

If you cat /sys/fs/cgroup/whatever/cgroup/cgroup.controllers and you see "io" in
there keep doing the above in the next highest parent directory until io is no
longer in there.  Thanks,

Josef