On 21.03.23 20:32, David Hildenbrand wrote:

On 20.03.23 22:27, Luis Chamberlain wrote:

quoted

On Mon, Mar 20, 2023 at 02:23:36PM -0700, Luis Chamberlain wrote:

quoted

On Mon, Mar 20, 2023 at 10:15:23PM +0100, David Hildenbrand wrote:

quoted

Not able to reproduce with 20230319-module-alloc-opts so far (2 tries).

Oh wow, so to clarify, it boots OK?

Now that we know that tree works, I'm curious also now if you can
confirm just re-ordering the patches still works (it should)

https://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git/log/?h=20230319-module-alloc-opts-adjust

So far no vmap errors booting the debug/kasan kernel (2 tries).

quoted

And although it's *probably* just noise, but I'm very curious how much,
if any difference there is if you just revert "module: use
list_add_tail_rcu() when adding module".

Dito, no vmap errors booting the debug/kasan kernel (2 tries).

quoted

The data on that commit log is pretty small as I have a low end system,
and I'm not yet done beating the hell out of a system with stress-ng,
but getting some data froma  pretty large system would be great.
Specially if this series seems to prove fixing boot on them.

2x booting RHEL9.1 on a !debug kernel. Something went wrong with kdump in 2 runs (think I
had to delete the kdump initrd to make space for another kernel), but we can just mostly
ignore that. I wanted to rerun with kdump disabled completely, but I'm out of time for today.


1) v6.3-rc1:

#1

Startup finished in 25.354s (kernel) + 7.662s (initrd) + 1min 8.805s (userspace) = 1min 41.822s
multi-user.target reached after 29.186s in userspace

47.178s kdump.service
14.898s tuned.service
11.394s chrony-wait.service
   7.486s systemd-udev-settle.service
   7.334s NetworkManager-wait-online.service
   2.908s initrd-switch-root.service
   2.451s smartd.service
   2.316s dracut-initqueue.service
   2.057s polkit.service
   1.290s NetworkManager.service
   1.046s cups.service
   ...

#2

Startup finished in 25.375s (kernel) + 7.497s (initrd) + 29.428s (userspace) = 1min 2.301s
multi-user.target reached after 29.392s in userspace

14.552s tuned.service
   9.410s chrony-wait.service
   8.126s systemd-udev-settle.service
   7.502s NetworkManager-wait-online.service
   2.871s initrd-switch-root.service
   2.401s kdump.service
   2.297s polkit.service
   2.116s dracut-initqueue.service
   2.027s smartd.service
   1.262s NetworkManager.service
   1.102s cups.service
   1.011s sshd.service
   ...


2) 20230319-module-alloc-opts-adjust + revert of list_add_tail_rcu():

#1

Startup finished in 25.441s (kernel) + 7.285s (initrd) + 1min 7.644s (userspace) = 1min 40.371s
multi-user.target reached after 27.213s in userspace

47.232s kdump.service
14.235s tuned.service
   8.220s chrony-wait.service
   7.444s NetworkManager-wait-online.service
   5.986s systemd-udev-settle.service
   2.881s initrd-switch-root.service
   2.236s smartd.service
   1.899s dracut-initqueue.service
   1.812s polkit.service
   1.554s NetworkManager.service
   1.140s ModemManager.service
   ...

#2

Startup finished in 25.377s (kernel) + 7.271s (initrd) + 28.247s (userspace) = 1min 897ms
multi-user.target reached after 28.210s in userspace

15.435s tuned.service
11.365s chrony-wait.service
   7.512s NetworkManager-wait-online.service
   5.962s systemd-udev-settle.service
   2.889s initrd-switch-root.service
   2.846s smartd.service
   2.819s kdump.service
   2.228s polkit.service
   1.872s dracut-initqueue.service
   1.312s NetworkManager.service
   1.152s ModemManager.service
   1.011s sshd.service
   ...

3) 20230319-module-alloc-opts-adjust:


#1

Startup finished in 25.320s (kernel) + 7.192s (initrd) + 1min 6.511s (userspace) = 1min 39.024s
multi-user.target reached after 28.205s in userspace

46.307s kdump.service
14.199s tuned.service
13.358s chrony-wait.service
   7.468s NetworkManager-wait-online.service
   6.329s systemd-udev-settle.service
   2.910s initrd-switch-root.service
   2.752s smartd.service
   2.142s polkit.service
   1.909s dracut-initqueue.service
   1.210s NetworkManager.service
   1.041s ModemManager.service
    925ms sshd.service
   ...

#2

Startup finished in 25.368s (kernel) + 7.303s (initrd) + 1min 6.897s (userspace) = 1min 39.569s
multi-user.target reached after 27.326s in userspace

45.626s kdump.service
14.707s tuned.service
13.246s chrony-wait.service
   7.428s NetworkManager-wait-online.service
   6.507s systemd-udev-settle.service
   3.038s initrd-switch-root.service
   3.001s smartd.service
   2.057s polkit.service
   1.746s dracut-initqueue.service
   1.221s NetworkManager.service
   ...


I think we primarily only care about systemd-udev-settle.service.

That is fastest without the rcu patch (~6s), compared to with the rcu
patch (~6.5s) and with stock (~7.5s -- 8s).

Looks like dracut-initqueue also might be a bit faster with your changes, but
maybe it's mostly noise (would have to do more runs).

So maybe drop that rcu patch? But of course, there could be other scenarios where it's
helpful ...

Are there any other things you would like me to measure/test? I'll have 
to hand back that test machine soonish.

-- 
Thanks,

David / dhildenb