qemu machine settings and root fs for armv8?

I was wondering if folks had recommendations for testing armv8 kernels
in qemu, such as what machine settings and if root fs images existed.
I've been playing around with buildroot, but was curious if people had
recipes they could share?
-- 
Thanks,
~Nick Desaulniers

On Wed, Aug 8, 2018 at 11:52 AM Nick Desaulniers
[off-list ref] wrote:

I was wondering if folks had recommendations for testing armv8 kernels
in qemu, such as what machine settings and if root fs images existed.
I've been playing around with buildroot, but was curious if people had
recipes they could share?

Following up, I found:
https://github.com/google/syzkaller/blob/master/docs/linux/setup_linux-host_qemu-vm_arm64-kernel.md
helpful to build the rootfs using buildroot.

Then:

$ qemu-system-aarch64 -kernel arch/arm64/boot/Image -machine virt -cpu
cortex-a57 -hda rootfs.ext2 -append "console=ttyAMA0 root=/dev/vda"
-serial mon:stdio -m 512 -nographic

worked for me.
-- 
Thanks,
~Nick Desaulniers

On 08/09/2018 01:10 AM, Nick Desaulniers wrote:

On Wed, Aug 8, 2018 at 11:52 AM Nick Desaulniers
[off-list ref] wrote:

quoted

I was wondering if folks had recommendations for testing armv8 kernels
in qemu, such as what machine settings and if root fs images existed.
I've been playing around with buildroot, but was curious if people had
recipes they could share?

Following up, I found:
https://github.com/google/syzkaller/blob/master/docs/linux/setup_linux-host_qemu-vm_arm64-kernel.md
helpful to build the rootfs using buildroot.

Then:

$ qemu-system-aarch64 -kernel arch/arm64/boot/Image -machine virt -cpu
cortex-a57 -hda rootfs.ext2 -append "console=ttyAMA0 root=/dev/vda"
-serial mon:stdio -m 512 -nographic

worked for me.

I usually use a buildroot environment and just boot as an initramfs:

qemu-system-aarch64 \
	-s \
	-smp 8 \
	-machine virt \
	-cpu cortex-a57 \
	-machine type=virt \
	-nographic \
	-m 1024 \
	-kernel /path/to/kernel \
	--append "nokaslr slub_debug=- console=ttyAMA0 debug_pagealloc=on 
earlycon=pl011,0x9000000" \
	-initrd /path/to/arm64_rootfs.cpio \
	-fsdev local,security_model=passthrough,id=fsdev0,path=/tmp/share \
	-device virtio-9p-pci,id=fs0,fsdev=fsdev0,mount_tag=hostshare

I'll note that this setup makes using kernel modules a big pain but
it's useful for certain types of testing.
The virtio settings are from https://www.linux-kvm.org/page/9p_virtio
for sharing files.

Thanks,
Laura

On Thu, Aug 9, 2018 at 12:36 AM Laura Abbott [off-list ref] wrote:

I usually use a buildroot environment and just boot as an initramfs:

I need to play around more with buildroot, but I'll bet that's an
option to make a ramdisk instead of a full disk image?

qemu-system-aarch64 \
        -s \
        -smp 8 \
        -machine virt \
        -cpu cortex-a57 \
        -machine type=virt \
        -nographic \
        -m 1024 \
        -kernel /path/to/kernel \
        --append "nokaslr slub_debug=- console=ttyAMA0 debug_pagealloc=on
earlycon=pl011,0x9000000" \

Very cool, thank you for sharing!

I'm guessing that kaslr makes it difficult to keep track of addresses
in between reboots?  There's a parameter "panic=-1" that will reboot
the kernel on panic that probably works well with this.

        -initrd /path/to/arm64_rootfs.cpio \
        -fsdev local,security_model=passthrough,id=fsdev0,path=/tmp/share \
        -device virtio-9p-pci,id=fs0,fsdev=fsdev0,mount_tag=hostshare

I'll note that this setup makes using kernel modules a big pain but

How come?

it's useful for certain types of testing.
The virtio settings are from https://www.linux-kvm.org/page/9p_virtio
for sharing files.

Wow, that's super cool!  That probably helps so that you don't need to
keep rebuilding your ramdisk.  For binaries I'd guess that any
dynamically linked dependencies would have to be in that dir, too,
otherwise the VM wouldn't be able to load them.  But maybe nice for
statically linked cross compiled binaries.

Thanks again.
-- 
Thanks,
~Nick Desaulniers

Hi Nick,

Nick Desaulniers writes:

On Thu, Aug 9, 2018 at 12:36 AM Laura Abbott [off-list ref] wrote:

quoted

I usually use a buildroot environment and just boot as an initramfs:

I need to play around more with buildroot, but I'll bet that's an
option to make a ramdisk instead of a full disk image?

The BR2_TARGET_ROOTFS_CPIO config option of Buildroot enables build of
.cpio archive which is suitable for use as initramfs/initrd. See under
'Filesystem images' -> 'cpio the root filesystem'.

In case you also build your kernel under Buildroot, then
BR2_TARGET_ROOTFS_CPIO would automatically make your kernel embed the
cpio initramfs (using the kernel CONFIG_INITRAMFS_SOURCE). With that, you
don't need to pass a separate -initrd option to qemu.

baruch

--
     http://baruch.siach.name/blog/                  ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch at tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il -

On 08/09/2018 07:15 PM, Nick Desaulniers wrote:

On Thu, Aug 9, 2018 at 12:36 AM Laura Abbott [off-list ref] wrote:

quoted

I usually use a buildroot environment and just boot as an initramfs:

I need to play around more with buildroot, but I'll bet that's an
option to make a ramdisk instead of a full disk image?

quoted

qemu-system-aarch64 \
         -s \
         -smp 8 \
         -machine virt \
         -cpu cortex-a57 \
         -machine type=virt \
         -nographic \
         -m 1024 \
         -kernel /path/to/kernel \
         --append "nokaslr slub_debug=- console=ttyAMA0 debug_pagealloc=on
earlycon=pl011,0x9000000" \

Very cool, thank you for sharing!

I'm guessing that kaslr makes it difficult to keep track of addresses
in between reboots?  There's a parameter "panic=-1" that will reboot
the kernel on panic that probably works well with this >

Yes, I just disable nokaslr for most testing to make looking
at the addresses easier since I'm often staring at kernel panics
and dumps.

quoted

         -initrd /path/to/arm64_rootfs.cpio \
         -fsdev local,security_model=passthrough,id=fsdev0,path=/tmp/share \
         -device virtio-9p-pci,id=fs0,fsdev=fsdev0,mount_tag=hostshare

I'll note that this setup makes using kernel modules a big pain but

How come?

I build the buildroot initrd separately from the kernel. This means that
to include modules in the initrd I have to rebuild the initrd which is
yet another step.

quoted

it's useful for certain types of testing.
The virtio settings are from https://www.linux-kvm.org/page/9p_virtio
for sharing files.

Wow, that's super cool!  That probably helps so that you don't need to
keep rebuilding your ramdisk.  For binaries I'd guess that any
dynamically linked dependencies would have to be in that dir, too,
otherwise the VM wouldn't be able to load them.  But maybe nice for
statically linked cross compiled binaries.

Yes, binaries are a bit of a pain. buildroot has a good selection of
libraries available to include so I can rebuild the ramdisk to include
those. It's still a lot of trial and error so if I can get away with
it I prefer to have everything statically linked.

Thanks again.

Thanks,
Laura