Create a disposable Azure VM based on CBL-Mariner

TL;DR

See how to

find VM image publisher, offer and SKU
use cloud-init and the dnf package manager with a CBL-Mariner VM for:
- Git
- Zsh
- Azure CLI
- .NET SDK 6.0
- Node.js
- Rust
- Go
- Python3 PIP
install from another package source
- Github CLI
- .NET SDK 3.1
install with original scripts
- Azure Developer CLI
- Oh-My-Zsh
install with bare installation steps
- kubectl CLI
- Azure Functions Core tools
- Docker

and generally, how to get certain packages installed for the non-root user.

Motivation

In a previous post I was already showing how I create a disposable VM. That one was based on Ubuntu 22.04 - practically a piece of cake as almost every corner of this distro is mentioned on the web.

Curious what this "Microsoft Linux" is about and to what degree it is already viable to be operated on my own VM I challenged myself to get such a disposable VM working with CBL-Mariner. As such there is no real added value for me running a VM with yet another distribution. It was just a pure learning exercise: I want to get familiar with Linux and hence - as I always do - go at it from different angles to have as much exposure as possible.

The Deployment Process

PowerShell script and Bicep templates are taken almost 1:1 from the other post.

Of course the VMs parameters are adjusted to the new distribution:

var vmImagePublisher = 'MicrosoftCBLMariner'
var vmImageOffer = 'cbl-mariner'
var vmImageSku = 'cbl-mariner-2-gen2'

Finding The VM Image

How did I get to these values above? I usually apply these steps:

I figured it would be an image published by "Microsoft". Hence I narrowed down all offerings by publisher name:

Get-AzVMImagePublisher -Location westeurope | ?{$_.PublisherName -match "Microsoft"} | Get-AzVMImageOffer

Scanning through this long list, the offer cbl-mariner and publisher MicrosoftCBLMariner looked promising.

Get-AzVMImageSku -Location westeurope -PublisherName MicrosoftCBLMariner -Offer cbl-mariner

From the various SKUs listed I go for cbl-mariner-2-gen2 ... gen2 is always better than the old ones, right?

With that I checked that there are actual images:

Get-AzVMImage -Location westeurope -PublisherName MicrosoftCBLMariner -Offer cbl-mariner -Skus cbl-mariner-2-gen2

The very same exercise can also be done with Azure CLI:

az vm image list-publishers -l westeurope --query "[?contains(name, 'Microsoft')]" -o table
az vm image list-offers -l westeurope -p MicrosoftCBLMariner -o table
az vm image list-skus -l westeurope -p MicrosoftCBLMariner -f cbl-mariner -o table
az vm image list -l westeurope -p MicrosoftCBLMariner -f cbl-mariner -s cbl-mariner-2-gen2 --all -o table

Cloud Init

First for better orientation the complete cloud-init.txt. Various packages or sections are explained below.

#cloud-config
write_files:
  - path: /usr/lib/systemd/system/docker.service
    content: | 
        [Unit]
        Description=Docker Application Container Engine
        Documentation=https://docs.docker.com
        After=network.target

        [Service]
        Type=notify
        # the default is not to use systemd for cgroups because the delegate issues still
        # exists and systemd currently does not support the cgroup feature set required
        # for containers run by docker
        ExecStart=/usr/bin/dockerd
        ExecReload=/bin/kill -s HUP $MAINPID
        # Having non-zero Limit*s causes performance problems due to accounting overhead
        # in the kernel. We recommend using cgroups to do container-local accounting.
        LimitNOFILE=infinity
        LimitNPROC=infinity
        LimitCORE=infinity
        # Uncomment TasksMax if your systemd version supports it.
        # Only systemd 226 and above support this version.
        #TasksMax=infinity
        TimeoutStartSec=0
        # set delegate yes so that systemd does not reset the cgroups of docker containers
        Delegate=yes
        # kill only the docker process, not all processes in the cgroup
        KillMode=process

        [Install]
        WantedBy=multi-user.target
  - path: /tmp/install-function-tools-non-root.sh
    content: | 
      #!/bin/bash
      mkdir -p ~/.local/bin
      npm config set prefix '~/.local/'
      echo 'export PATH=~/.local/bin/:$PATH' >> ~/.bashrc
      npm install --location=global azure-functions-core-tools@4
    permissions: '0755'
  - path: /tmp/install-oh-my-zsh.sh
    content: | 
      #!/bin/bash
      sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)" "" --unattended
    permissions: '0755'
runcmd:
- export USER=$(awk -v uid=1000 -F":" '{ if($3==uid){print $1} }' /etc/passwd)

- dnf upgrade

- dnf install less git zsh -y
- sudo -H -u $USER bash -c '/tmp/install-oh-my-zsh.sh'

- dnf install 'dnf-command(config-manager)' -y
- dnf config-manager --add-repo https://cli.github.com/packages/rpm/gh-cli.repo
- dnf install gh -y

- dnf install azure-cli -y
- curl -fsSL https://aka.ms/install-azd.sh | bash
- az extension add --name containerapp --upgrade 
- az bicep install

- curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
- install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

- dnf install dotnet-sdk-6.0 -y
- dnf install nodejs -y
- sudo -H -u $USER bash -c '/tmp/install-function-tools-non-root.sh'

- rpm --import https://packages.microsoft.com/keys/microsoft.asc
- wget -q -O /etc/yum.repos.d/microsoft-prod.repo https://packages.microsoft.com/config/fedora/36/prod.repo
- dnf install dotnet-sdk-3.1 -y

- dnf install rust golang python3-pip -y

- wget -q -O docker.tgz https://download.docker.com/linux/static/stable/x86_64/docker-20.10.9.tgz
- tar xzvf docker.tgz
- mv docker/* /usr/bin/
- systemctl enable docker
- systemctl start docker

User Context

This first command determines the username of the non-root user used in the installation.

runcmd:
- export USER=$(awk -v uid=1000 -F":" '{ if($3==uid){print $1} }' /etc/passwd)

Package Manager

CBL-Mariner does not come with apt - hence these commands of my regular cloud-init.txt

package_upgrade: true
packages:
- apt-transport-https
- ...

do not work and everything has to be installed using dnf

- dnf upgrade

- dnf install less git zsh -y

Oh-My-Zsh

One of the cases where installation with non-root user is required (to work properly later) - initiated by

- sudo -H -u $USER bash -c '/tmp/install-oh-my-zsh.sh'

and handled by the script file placed in the upper section

  - path: /tmp/install-oh-my-zsh.sh
    content: | 
      #!/bin/bash
      sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)" "" --unattended
    permissions: '0755'

my tests revealed that the sh -c "$(curl... command only gets the user context when executed within that script, not when directly executed with - sudo -H -u $USER sh -c "$(curl...

GitHub CLI

For gh another package repository needs to be added using the DNF config-manager Plugin:

- dnf install 'dnf-command(config-manager)' -y
- dnf config-manager --add-repo https://cli.github.com/packages/rpm/gh-cli.repo
- dnf install gh -y

Azure CLI and Azure Developer CLI

For az there is a standard package - for azd the provided script can be used:

- dnf install azure-cli -y
- curl -fsSL https://aka.ms/install-azd.sh | bash
- az extension add --name containerapp --upgrade 
- az bicep install

kubectl

Bare installation steps:

- curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
- install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

Programming languages and Azure Function Core Tools

It seems that everything that is supposedly required for Microsoft's own downstream VMs is packaged, other more auxiliary things are not. For .NET SDK 3.1 I pulled in the currently latest Fedora repo.

- dnf install dotnet-sdk-6.0 -y
- dnf install nodejs -y
- sudo -H -u $USER bash -c '/tmp/install-function-tools-non-root.sh'

- rpm --import https://packages.microsoft.com/keys/microsoft.asc
- wget -q -O /etc/yum.repos.d/microsoft-prod.repo https://packages.microsoft.com/config/fedora/36/prod.repo
- dnf install dotnet-sdk-3.1 -y

- dnf install rust golang python3-pip -y

Again, Azure Function Core Tools need to be installed in non-root user context with a script:

  - path: /tmp/install-function-tools-non-root.sh
    content: | 
      #!/bin/bash
      mkdir -p ~/.local/bin
      npm config set prefix '~/.local/'
      echo 'export PATH=~/.local/bin/:$PATH' >> ~/.zshrc
      npm install --location=global azure-functions-core-tools@4
    permissions: '0755'

Docker

Docker currently needs a really bare installation process - none of the regular scripts did work:

Element 1 - installation

- wget -q -O docker.tgz https://download.docker.com/linux/static/stable/x86_64/docker-20.10.9.tgz
- tar xzvf docker.tgz
- mv docker/* /usr/bin/
- systemctl enable docker
- systemctl start docker

Element 2 - systemd Docker service definition

  - path: /usr/lib/systemd/system/docker.service
    content: | 
        [Unit]
        Description=Docker Application Container Engine
        Documentation=https://docs.docker.com
        After=network.target

        [Service]
        Type=notify
        # the default is not to use systemd for cgroups because the delegate issues still
        # exists and systemd currently does not support the cgroup feature set required
        # for containers run by docker
        ExecStart=/usr/bin/dockerd
        ExecReload=/bin/kill -s HUP $MAINPID
        # Having non-zero Limit*s causes performance problems due to accounting overhead
        # in the kernel. We recommend using cgroups to do container-local accounting.
        LimitNOFILE=infinity
        LimitNPROC=infinity
        LimitCORE=infinity
        # Uncomment TasksMax if your systemd version supports it.
        # Only systemd 226 and above support this version.
        #TasksMax=infinity
        TimeoutStartSec=0
        # set delegate yes so that systemd does not reset the cgroups of docker containers
        Delegate=yes
        # kill only the docker process, not all processes in the cgroup
        KillMode=process

        [Install]
        WantedBy=multi-user.target

Fascinating: With this installation no addition of non-root user to group docker is required.

Conclusion

As CBL-Mariner definitely can be considered "niche" there is almost no direct documentation available. Everything shown above is stitched together looking for completely manual / bare installation steps or DNF/RPM based installation. Anyway, that was a nice experience for me to get me out of my apt and pacman comfort zone.