In this blog post, we will look at an example of how to use Azure Disk as a storage medium for your apps deployed to Azure Kubernetes Service.

You will:

• Setup a Kubernetes cluster on Azure
• Create an Azure Disk and a corresponding PersistentVolume
• Create a PersistentVolumeClaim for the app Deployment
• Test things out to see how it all works end to end

Overview

You can use Kubernetes Volumes to provide storage for your applications. There is support for multiple types of volumes in Kubernetes. One way of categorizing them is as follows

• Ephemeral - Volumes which are tightly coupled with the Pod lifetime (e.g. emptyDir volume) i.e. they are deleted if the Pod is removed (for any reason).
• Persistent - Volumes which are meant for long term storage and independent of the Pod or the Node lifecycle. This could be NFS or cloud based storage in case of managed Kubernetes offerings such as Azure Kubernetes Service, Google Kubernetes Engine etc.

Kubernetes Volumes can be provisioned in a static or dynamic manner. In "static" mode, the storage medium e.g. Azure Disk, is created manually and then referenced using the Pod spec as below:

    volumes:
        - name: azure
            azureDisk:
            kind: Managed
            diskName: myAKSDisk
            diskURI: /subscriptions/<subscriptionID>/resourceGroups/MC_myAKSCluster_myAKSCluster_eastus/providers/Microsoft.Compute/disks/myAKSDisk

I would highly recommend reading through the excellent tutorial on how to "Manually create and use a volume with Azure disks in Azure Kubernetes Service (AKS)"

Is there a better way?

In the above Pod manifest, the storage info is directly specified in the Pod (using the volumes section). This implies that the developer needs to know all details of the storage medium e.g. in case of Azure Disk - the diskName, diskURI (disk resource URI), it's kind (type). There is definitely scope for improvement here and like most things in software, it can be done with another level of indirection or abstraction using concepts of Persistent Volume and Persistent Volume Claim.

The key idea revolves around "segregation of duties" and decoupling storage creation/management from its usage:

• When an app needs persistent storage for their application, the developer can request for it by "declaring" it in the pod spec - this is done using a PersistentVolumeClaim
• The actual storage provisioning e.g. creation of Azure Disk (using azure CLI, portal etc.) and representing it in the Kubernetes cluster (using a PersistentVolume) can be done by another entity such as an admin

Let's see this in action!

Pre-requisites:

You will need the following:

• Microsoft Azure account - go ahead and sign up for a free one!
• Azure Kubernetes Service (AKS) cluster - this blog will guide you through the process of creating one
• Azure CLI or Azure Cloud Shell - you can either choose to install the Azure CLI if you don't have it already (should be quick!) or just use the Azure Cloud Shell from your browser.
• kubectl to interact with yur AKS cluster

on your Mac, you can install kubectl as such:

curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/darwin/amd64/kubectl
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin/kubectl

The code is available on GitHub. Please clone the repository before you proceed

git clone https://github.com/abhirockzz/aks-azuredisk-static-pv
cd aks-azuredisk-static-pv

Kubernetes cluster setup

You need a single command to stand up a Kubernetes cluster on Azure. But, before that, we'll have to create a resource group

export AZURE_SUBSCRIPTION_ID=[to be filled]
export AZURE_RESOURCE_GROUP=[to be filled]
export AZURE_REGION=[to be filled] (e.g. southeastasia)

Switch to your subscription and invoke az group create

az account set -s $AZURE_SUBSCRIPTION_ID
az group create -l $AZURE_REGION -n $AZURE_RESOURCE_GROUP

You can now invoke az aks create to create the new cluster

To keep things simple, the below command creates a single node cluster. Feel free to change the specification as per your requirements

export AKS_CLUSTER_NAME=[to be filled]

az aks create --resource-group $AZURE_RESOURCE_GROUP --name $AKS_CLUSTER_NAME --node-count 1 --node-vm-size Standard_B2s --node-osdisk-size 30 --generate-ssh-keys

Get the AKS cluster credentials using az aks get-credentials - as a result, kubectl will now point to your new cluster. You can confirm the same

az aks get-credentials --resource-group $AZURE_RESOURCE_GROUP --name $AKS_CLUSTER_NAME
kubectl get nodes

If you are interested in learning Kubernetes and Containers using Azure, a good starting point is to use the quickstarts, tutorials and code samples in the documentation to familiarize yourself with the service. I also highly recommend checking out the 50 days Kubernetes Learning Path. Advanced users might want to refer to Kubernetes best practices or the watch some of the videos for demos, top features and technical sessions.

Create an Azure Disk for persistent storage

An Azure Kubernetes cluster can use Azure Disks or Azure Files as data volumes. In this example, we will explore Azure Disk. You have the option of an Azure Disk backed by a Standard HDD or a Premium SSD

Get the AKS node resource group

AKS_NODE_RESOURCE_GROUP=$(az aks show --resource-group $AZURE_RESOURCE_GROUP --name $AKS_CLUSTER_NAME --query nodeResourceGroup -o tsv)

Create an Azure Disk in the node resource group

export AZURE_DISK_NAME=<enter-azure-disk-name>

az disk create --resource-group $AKS_NODE_RESOURCE_GROUP --name $AZURE_DISK_NAME --size-gb 2 --query id --output tsv

we are creating a Disk with a capacity of 2 GB

You will get the resource ID of the Azure Disk as a response which will be used in the next step

/subscriptions/3a06a10f-ae29-4242-b6a7-dda0ea91d342/resourceGroups/MC_testaks_foo-aks_southeastasia/providers/Microsoft.Compute/disks/my-test-disk

Deploy the app to Kubernetes

The azure-disk-persistent-volume.yaml file contains the PersistentVolume details. We create it in order to map the Azure Disk within the AKS cluster.

Notice that the capacity (spec.capacity.storage) is 2 GB which is same as that of the Azure Disk we just created

Update azure-disk-persistent-volume.yaml with Azure Disk info

• diskName - name of the Azure Disk which you chose earlier
• diskURI - resource ID of the Azure Disk

Create the PersistentVolume

kubectl apply -f azure-disk-persistent-volume.yaml

persistentvolume/azure-disk-pv created

Next we need to create the PersistentVolumeClaim which we will use as a reference in the Pod specification

we are requesting for 2 GB worth of storage (using resources.request.storage)

To create it:

kubectl apply -f azure-disk-persistent-volume-claim.yaml

persistentvolumeclaim/azure-disk-pvc created

Check the PersistentVolume

kubectl get pv/azure-disk-pv


NAME            CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM    STORAGECLASS   REASON   AGE
azure-disk-pv   2Gi        RWO            Retain           Bound    default/azure-disk-pvc            8m35s

Check the PersistentVolumeClaim

kubectl get pvc/azure-disk-pvc

NAME             STATUS   VOLUME          CAPACITY   ACCESS MODES   STORAGECLASS   AGE
azure-disk-pvc   Bound    azure-disk-pv   2Gi        RWO                           9m55s

Notice (in the STATUS section of the above outputs) that the PersistentVolume and PersistentVolumeClaim are Bound to each other

Test

To test things out, we will use a simple Go app. All it does is push log statements to a file logz.out in /mnt/logs - this is the path which is mounted into the Pod

func main() {
    ticker := time.NewTicker(3 * time.Second)
    exit := make(chan os.Signal, 1)
    signal.Notify(exit, syscall.SIGTERM, syscall.SIGINT)
    for {
        select {
        case t := <-ticker.C:
            logToFile(t.String())
        case <-exit:
            err := os.Remove(fileLoc + fileName)
            if err != nil {
                log.Println("unable to delete log file")
            }
            os.Exit(1)
        }
    }
}

To create our app as a Deployment

kubectl apply -f app-deployment.yaml

Wait for a while for the deployment to be in Running state

kubectl get pods -l=app=logz

NAME                               READY   STATUS    RESTARTS   AGE
logz-deployment-59b75bc786-wt98d   1/1     Running   0          15s

To confirm, check the /mnt/logs/logz.out in the Pod

kubectl exec -it $(kubectl get pods -l=app=logz --output=jsonpath={.items..metadata.name}) -- tail -f /mnt/logs/logz.out

You will see the logs (just the timestamp) every 3 seconds. This is because the Azure Disk storage has been mounted inside your Pod

2019-09-23 10:00:18.308746334 +0000 UTC m=+3.002071866
2019-09-23 10:00:21.308779348 +0000 UTC m=+6.002104880
2019-09-23 10:00:24.308771261 +0000 UTC m=+9.002096693
2019-09-23 10:00:27.308778874 +0000 UTC m=+12.002104406
2019-09-23 10:00:30.308804587 +0000 UTC m=+15.002130219

Once you're done testing, you can delete the resources to save costs.

To clean up

az group delete --name $AZURE_RESOURCE_GROUP --yes --no-wait

This will delete all resources under the resource group

That's all for this blog! You saw how to attach and mount an Azure Disk instance to your app running in AKS using standard Kubernetes primitives like PersistentVolume and PersistentVolume. Stay tuned for more 😃😃

I really hope you enjoyed and learned something from this article! Please like and follow if you did. Happy to get feedback via Twitter or just drop a comment