## Intro Sometimes, testing your work on the Raspberry Pi OS is much easier without running it on real hardware. Things like Ansible Playbooks or a Kubernetes cluster, in most cases, can be tested in a virtualized environment. There are plenty of tutorials and other Docker images running Raspberry Pi OS using [QEMU](https://www.qemu.org/), but unfortunately, all of them utilize the OS image at runtime as an SD card. Hence, they do not support mounting volumes to share the filesystem. ## Image The most obvious solution would be to extract the OS image and share the extracted folder with the virtual machine. Fortunately, QEMU provides such an option out of the box with the `-virtfs` flag: ``` -virtfs local,path=path,mount_tag=mount_tag ,security_model=security_model[,writeout=writeout][,readonly=on] [,fmode=fmode][,dmode=dmode][,multidevs=multidevs] ``` Let's now try to download and extract all the OS files from a Raspberry Pi OS distribution: ```bash curl https://downloads.raspberrypi.com/raspios_lite_arm64/images/raspios_lite_arm64-2024-10-28/2024-10-22-raspios-bookworm-arm64-lite.img.xz \ | unxz -c - >/tmp/sd.img ``` The extracted image can be inspected with the [`fdisk`](https://linux.die.net/man/8/fdisk) command: ```bash $ fdisk -l /tmp/sd.img Disk /tmp/sd.img: 2732 MB, 2864709632 bytes, 5595136 sectors 43712 cylinders, 4 heads, 32 sectors/track Units: sectors of 1 * 512 = 512 bytes Device Boot StartCHS EndCHS StartLBA EndLBA Sectors Size Id Type /tmp/sd.img1 64,0,1 1023,3,32 8192 1056767 1048576 512M c Win95 FAT32 (LBA) /tmp/sd.img2 1023,3,32 1023,3,32 1056768 5595135 4538368 2216M 83 Linux ``` As we can see, the image has two partitions. The first one, the boot partition, is of type `FAT32`, and the second is of type `ext4`. We can mount those partitions using the [`loop`](https://en.wikipedia.org/wiki/Loop_device#Uses_of_loop_mounting) device with [`mount`](https://linux.die.net/man/8/mount), [`sfdisk`](https://linux.die.net/man/8/sfdisk), and [`jq`](https://jqlang.github.io/jq/) in one line: ```bash mount -o loop,offset="$(sfdisk -J /tmp/sd.img | jq '.partitiontable.sectorsize * .partitiontable.partitions[1].start')" /tmp/sd.img /tmp/sd ``` Combining those all together in a `Dockerfile` would look something like this: ```Dockerfile # syntax=docker/dockerfile:1.3-labs FROM alpine:latest AS image ADD https://downloads.raspberrypi.com/raspios_lite_arm64/images/raspios_lite_arm64-2024-10-28/2024-10-22-raspios-bookworm-arm64-lite.img.xz /sd.img.xz RUN --security=insecure \ apk add --no-cache --virtual=.tools \ jq \ sfdisk \ && unxz -ck /sd.img.xz >/tmp/sd.img \ && mkdir -p /tmp/sd \ && mount -o loop,offset="$(sfdisk -J /tmp/sd.img | jq '.partitiontable.sectorsize * .partitiontable.partitions[1].start')" /tmp/sd.img /tmp/sd \ && mount -o loop,offset="$(sfdisk -J /tmp/sd.img | jq '.partitiontable.sectorsize * .partitiontable.partitions[0].start')" /tmp/sd.img /tmp/sd/boot/firmware \ && cp -pr /tmp/sd /media/sd \ && umount /tmp/sd/boot/firmware /tmp/sd \ && rm -rf /tmp/sd /tmp/sd.img \ && apk del .tools ``` Mounting volumes requires a privileged mode. That is why we need the `--security=insecure` flag after the `RUN` instruction, which is only available in the labs specification. ## Kernel Now, if we try to run the extracted image using QEMU: ```bash qemu-system-aarch64 \ -serial mon:stdio \ -nographic \ -no-reboot \ -machine virt \ -cpu cortex-a72 \ -m 1G \ -smp 4 \ -device virtio-net-device,netdev=net0 \ -netdev user,id=net0 \ -kernel /media/sd/boot/firmware/kernel8.img \ -virtfs local,id=boot,mount_tag=boot,multidevs=remap,path=/media/sd/boot/firmware,security_model=none \ -virtfs local,id=root,mount_tag=root,multidevs=remap,path=/media/sd,security_model=none \ -append "console=ttyAMA0,115200 root=root rootflags=cache=mmap,msize=104857600,posixacl,trans=virtio,version=9p2000.L rootfstype=9p rootwait" ``` The kernel will throw a panic since it cannot mount the root of type `9p`: ``` [ 1.144617] Disabling rootwait; root= is invalid. [ 1.153539] VFS: Cannot open root device "root" or unknown-block(0,0): error -19 [ 1.154459] Please append a correct "root=" boot option; here are the available partitions: ... [ 1.156259] List of all bdev filesystems: [ 1.156335] ext3 [ 1.156346] ext2 [ 1.156386] ext4 [ 1.156414] vfat [ 1.156440] msdos [ 1.156469] f2fs [ 1.156497] [ 1.156625] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0) ``` Even though the `9pfs` support is present in the Linux kernel, the Raspberry Pi OS [kernel](https://github.com/raspberrypi/linux) was not built with the corresponding [flags](https://wiki.qemu.org/Documentation/9psetup#Preparation). Let's try to fix it by rebuilding the kernel ourselves: ```Dockerfile FROM ubuntu:latest AS kernel ENV ARCH=arm64 ENV CROSS_COMPILE=aarch64-linux-gnu- WORKDIR /tmp/kernel ADD https://github.com/raspberrypi/linux/archive/refs/tags/stable_20241008.tar.gz /kernel.tar.gz RUN tar --strip-components=1 -xzf /kernel.tar.gz \ && apt-get update \ && apt-get install --mark-auto --no-install-recommends -y \ bc \ bison \ flex \ gcc \ gcc-aarch64-linux-gnu \ libc6-dev \ libc6-dev-arm64-cross \ libssl-dev \ make \ && make O=/tmp/build defconfig \ && scripts/config --file /tmp/build/.config \ --set-val CONFIG_9P_FS y \ --set-val CONFIG_9P_FS_POSIX_ACL y \ --set-val CONFIG_9P_FS_SECURITY y \ --set-val CONFIG_NETWORK_FILESYSTEMS y \ --set-val CONFIG_NET_9P y \ --set-val CONFIG_NET_9P_VIRTIO y \ --set-val CONFIG_PCI y \ --set-val CONFIG_PCI_HOST_COMMON y \ --set-val CONFIG_PCI_HOST_GENERIC y \ --set-val CONFIG_VIRTIO_PCI y \ --set-val CONFIG_VIRTIO_BLK y \ --set-val CONFIG_VIRTIO_NET y \ && make O=/tmp/build -j 3 Image.gz \ && rm -rf /tmp/kernel \ && mkdir -p /media/sd/boot/firmware \ && cp /tmp/build/arch/$ARCH/boot/Image.gz /media/sd/boot/firmware/kernel8.img ``` ## Configuration Now, the command above should boot the operating system, which still fails to initialize completely: ``` [FAILED] Failed to start systemd-re…ount Root and Kernel File Systems. See 'systemctl status systemd-remount-fs.service' for details. [ TIME ] Timed out waiting for device /dev/disk/by-partuuid/385cce61-01. ``` This is failing due to incorrect records in `/etc/fstab` as we provide the filesystem over `9pfs`. That can be fixed by overwriting the prebaked `/etc/fstab`: ```Dockerfile COPY </tmp/sd.img \ && mkdir -p /tmp/sd \ && mount -o loop,offset="$(sfdisk -J /tmp/sd.img | jq '.partitiontable.sectorsize * .partitiontable.partitions[1].start')" /tmp/sd.img /tmp/sd \ && mount -o loop,offset="$(sfdisk -J /tmp/sd.img | jq '.partitiontable.sectorsize * .partitiontable.partitions[0].start')" /tmp/sd.img /tmp/sd/boot/firmware \ && cp -pr /tmp/sd /media/sd \ && umount /tmp/sd/boot/firmware /tmp/sd \ && rm -rf /tmp/sd /tmp/sd.img \ && apk del .tools RUN rm \ /media/sd/etc/init.d/resize2fs_once \ /media/sd/etc/systemd/system/multi-user.target.wants/rpi-eeprom-update.service COPY <

dokmic

Running Raspberry Pi OS in a Docker Container

Join Our Newsletter. No Spam, Only the good stuff.