My latest project has been to re-create my homelab using cloud native principles such as Infrastructure-as-Code and Dev(Git)Ops.

After much research and deliberation, I've concluded the best way for me to achieve this is to separate my local hardware from the software which will run on it. Essentially, I'll be treating my hardware as a "local cloud", and create VM Images that I will deploy running my application stack. (Azure Stack HCI has the right idea here!)

When developing and testing on cloud providers, they always have a base image available to use as a starting off point. Not so for my local cloud!

In this article, I'll demonstrate how to create a cloud-init enabled Ubuntu 20.04 LTS base image to use on Proxmox VE.

Cloud Native Image

There are several guides out there which utilize the "proxmox-iso" packer builder to build a base image from an ISO file. Modern Linux distributions are increasingly moving away from this install method and preseed files. Rather, disk images are provided with the OS pre-installed, and configuration is performed via cloud-init. We will create a Proxmox KVM base image using Ubuntu's KVM cloud image.

NOTE: Since the Packer Proxmox builder doesn't support LXD images nor templates, we need to create a KVM template.

Proxmox Script

The Proxmox API doesn't appear to offer the full functionality provided by the native shell commands to create a template, so we will run a script via SSH.
This gist contains the script, and I'll step through it below. https://gist.github.com/mike1237/cce83a74f898b11c2cec911204568cf9

Download the image

We are downloading the kvm disk image. Note that this is a qcow2 image format with an extension of .img
Promxox doesn't like this so we rename the disk image to .qcow2

SRC_IMG="https://cloud-images.ubuntu.com/focal/current/focal-server-cloudimg-amd64-disk-kvm.img"
IMG_NAME="focal-server-cloudimg-amd64-disk-kvm.qcow2"
wget -O $IMG_NAME $SRC_IMG

Add QEMU Guest Agent

The Ubuntu 20.04 image we are going to use does not include the qemu-guest-agent package which is needed for the Guest VM to report its IP details back to Proxmox. This is required for Packer to communicate with the VM after cloning. the template. libguestfs-tools will allow us to embed qemu-guest-agent into the image. You can also add any additional packages you'd like in your base image. Personally, I prefer to customize this base image later with packer so that the packages can live in source control.

apt update
apt install -y libguestfs-tools
virt-customize --install qemu-guest-agent -a $IMG_NAME

Create a VM in Proxmox with required settings and convert to template

For best performance, virtio 'hardware' should be used. Additionally, cloud-init requires a serial console and cloudinit IDE (CDROM) drive. We will set the network config to DHCP so that we get an IP address. Lastly, we will expand the template disk image size so we have space to install items later. It appears packer doesn't support doing this later.

TEMPL_NAME="ubuntu2004-cloud"
VMID="9000"
MEM="512"
DISK_SIZE="32G"
DISK_STOR="local-lvm"
NET_BRIDGE="vmbr0"
qm create $VMID --name $TEMPL_NAME --memory $MEM --net0 virtio,bridge=$NET_BRIDGE
qm importdisk $VMID $IMG_NAME $DISK_STOR
qm set $VMID --scsihw virtio-scsi-pci --scsi0 $DISK_STOR:vm-$VMID-disk-0
qm set $VMID --ide2 $DISK_STOR:cloudinit
qm set $VMID --boot c --bootdisk scsi0
qm set $VMID --serial0 socket --vga serial0
qm set $VMID --ipconfig0 ip=dhcp
qm resize $VMID scsi0 $DISK_SIZE
qm template $VMID
# Remove downloaded image
rm $IMG_NAME

Packer template

Now that we have our cloud-init enabled image on Proxmox, we can use Packer to create a template based off of this template.
Ensure to set the scsi_controller="virtio-scsi-pci" and qemu_agent=true.

I'd recommend adding the Proxmox variables to a var file.

packer build --var-file=./proxmox.pkvars.hcl --var "proxox_template_name=test-output-template" --var "proxmox_source_template=ubuntu2004-cloud" base.pkr.hcl

Next Steps

Now that you've created a template using packer from the base template, you can use Terraform to deploy that VM!
I'm still working on that myself :)

References

https://gist.github.com/chriswayg/43fbea910e024cbe608d7dcb12cb8466
https://whattheserver.com/proxmox-cloud-init-os-template-creation/
https://norocketscience.at/deploy-proxmox-virtual-machines-using-cloud-init/
https://pve.proxmox.com/wiki/Cloud-Init_Support
https://blog.dustinrue.com/2020/05/going-deeper-with-proxmox-cloud-init/