TheLazy.Dev
Portfolio GitHub

Blog

kubernetes |showdev |tutorial |beginners

How to use custom Storage Classes in Kubernetes

abhirockzz

Abhishek Gupta

Posted on September 30, 2019

How to use custom Storage Classes in Kubernetes

Most Kubernetes offerings provide default Storage Classes out of the box to ease the process of dynamic storage provisioning. But, what if you have different requirements that are not satisfied by the default ones?

In this article, we will look at how you can create your own Storage Class to meet your needs. The example scenario is based on an Azure Kubernetes Service but should be applicable to any Kubernetes provider.

code is available on GitHub

"The definitive guide to Kubernetes Volumes (Part 2)" blog post explored Kubernetes Storage Classes along with an example that leverages the default StorageClass. To trigger Dynamic Provisioning using the default Storage Class in your Kubernetes cluster, simply exclude the storageClass attribute from your PersistentVolumeClaim. For example, Azure Kubernetes Service includes two pre-seeded storage classes,

You can check the same by running kubectl get storageclass command

NAME                PROVISIONER                AGE
default (default)   kubernetes.io/azure-disk   6d10h
managed-premium     kubernetes.io/azure-disk   6d10h
Enter fullscreen mode Exit fullscreen mode

Here is what the default storage class spec looks like (this is a trimmed version obtained by executing kubectl get sc/default on my AKS cluster)

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  labels:
    kubernetes.io/cluster-service: "true"
  name: default
parameters:
  cachingmode: ReadOnly
  kind: Managed
  storageaccounttype: Standard_LRS
provisioner: kubernetes.io/azure-disk
reclaimPolicy: Delete
volumeBindingMode: Immediate
Enter fullscreen mode Exit fullscreen mode

OK, so what?

I highlighted this because I want you to take note of the following attributes, their values and what they mean

  • volumeBindingMode: Immediate - This setting implies that the PersistentVolume creation, followed with the storage medium (Azure Disk in this case) provisioning is triggered as soon as the PersistentVolumeClaim is created.
  • reclaimPolicy: Delete - With this setting, as soon as a PersistentVolumeClaim is deleted, it also triggers the removal of the corresponding PersistentVolume along with the Azure Disk. You are in for a surprise if you intended to retain that data for backup, other applications, etc.

Both reclaimPolicy: Delete and volumeBindingMode: Immediate are default settings

Override volumeBindingMode

Valid values include Immediate and WaitForFirstConsumer. Overriding the default value with WaitForFirstConsumer will delay the binding and provisioning of a PersistentVolume until a Pod (aka your app) using the PersistentVolumeClaim is created.

Override reclaimPolicy

Overriding the default value to Retain still delete the PersistentVolume after PersistentVolumeClaim is removed, but ensures that the actual storage medium is not purged.

note that Recycle is a deprecated option for reclaimPolicy

Hands-on: create and use a custom storage class

To follow along, you will need:

Kubernetes cluster setup

You need a single command to stand up a Kubernetes cluster on Azure. But, before that, we'll have to create a resource group

export AZURE_SUBSCRIPTION_ID=[to be filled]
export AZURE_RESOURCE_GROUP=[to be filled]
export AZURE_REGION=[to be filled] (e.g. southeastasia)
Enter fullscreen mode Exit fullscreen mode

Switch to your subscription and invoke az group create

az account set -s $AZURE_SUBSCRIPTION_ID
az group create -l $AZURE_REGION -n $AZURE_RESOURCE_GROUP
Enter fullscreen mode Exit fullscreen mode

You can now invoke az aks create to create the new cluster

To keep things simple, the below command creates a single node cluster. Feel free to change the specification as per your requirements

export AKS_CLUSTER_NAME=[to be filled]

az aks create --resource-group $AZURE_RESOURCE_GROUP --name $AKS_CLUSTER_NAME --node-count 1 --node-vm-size Standard_B2s --node-osdisk-size 30 --generate-ssh-keys
Enter fullscreen mode Exit fullscreen mode

Get the AKS cluster credentials using az aks get-credentials - as a result, kubectl will now point to your new cluster. You can confirm the same

az aks get-credentials --resource-group $AZURE_RESOURCE_GROUP --name $AKS_CLUSTER_NAME
kubectl get nodes
Enter fullscreen mode Exit fullscreen mode

If you are interested in learning Kubernetes and Containers using Azure, a good starting point is to use the quickstarts, tutorials and code samples in the documentation to familiarize yourself with the service. I also highly recommend checking out the 50 days Kubernetes Learning Path. Advanced users might want to refer to Kubernetes best practices or the watch some of the videos for demos, top features, and technical sessions.

Test it out...

Create the storage class and confirm its created

    kubectl apply -f https://raw.githubusercontent.com/abhirockzz/kubernetes-custom-storageclass/master/custom-storage-class.yaml
    kubectl get sc
Enter fullscreen mode Exit fullscreen mode

To keep things simple, the YAML file is being referenced directly from the GitHub repo, but you can also download the file to your local machine and use it in the same way.

You should see the new Storage Class

NAME                            PROVISIONER                AGE
azuredisk-custom-storageclass   kubernetes.io/azure-disk   4s
default (default)               kubernetes.io/azure-disk   2d10h
managed-premium                 kubernetes.io/azure-disk   2d10h
Enter fullscreen mode Exit fullscreen mode

Create the PersistentVolumeClaim

    kubectl apply -f https://raw.githubusercontent.com/abhirockzz/kubernetes-custom-storageclass/master/pvc.yaml
    kubectl get pvc/app-pvc
Enter fullscreen mode Exit fullscreen mode

You will notice that the STATUS is Pending which means that it is not yet associated with a PersistentVolume.

NAME      STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS                    AGE
app-pvc   Pending                                      azuredisk-custom-storageclass   10s
Enter fullscreen mode Exit fullscreen mode

Create a Deployment (will create a Pod)

    kubectl apply -f https://raw.githubusercontent.com/abhirockzz/kubernetes-custom-storageclass/master/deployment.yaml
Enter fullscreen mode Exit fullscreen mode

Pod will transition to the Running state in some time

Once it's Running, check PersistentVolumeClaim again

    kubectl get pvc/app-pvc
Enter fullscreen mode Exit fullscreen mode

The STATUS would have changed to Bound. As a result of the Pod creation, the storage request was analyzed and dynamic provisioning was triggered using the StorageClass mentioned in the PersistentVolumeClaim (azuredisk-custom-storageclass). As a result, an Azure Disk was provisioned along with the creation of a PersistentVolume and its association to the PersistentVolumeClaim. 

NAME      STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS                    AGE
app-pvc   Bound    pvc-83020049-e107-11e9-93ab-025752f370d3   1Gi        RWO            azuredisk-custom-storageclass   4m53s
Enter fullscreen mode Exit fullscreen mode

All this happened after our app was deployed, thanks to the WaitForFirstConsumer setting for volumeBindingMode property.

Confirm the same using kubectl get pv

NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                    STORAGECLASS                    REASON   AGE
pvc-83020049-e107-11e9-93ab-025752f370d3   1Gi        RWO            Retain           Bound    default/app-pvc          azuredisk-custom-storageclass            4m46s
Enter fullscreen mode Exit fullscreen mode

And of course, an Azure Disk will also be created. To check,

AKS_NODE_RESOURCE_GROUP=$(az aks show --resource-group $AZURE_RESOURCE_GROUP --name $AKS_CLUSTER_NAME --query nodeResourceGroup -o tsv)

az disk list -g $AKS_NODE_RESOURCE_GROUP
Enter fullscreen mode Exit fullscreen mode

The tags section will look something similar to

"tags": {
      "created-by": "kubernetes-azure-dd",
      "kubernetes.io-created-for-pv-name": "pvc-83020049-e107-11e9-93ab-025752f370d3",
      "kubernetes.io-created-for-pvc-name": "app-pvc",
      "kubernetes.io-created-for-pvc-namespace": "default"
}
Enter fullscreen mode Exit fullscreen mode

Let's delete the app and the PersistentVolumeClaim as well.

kubectl delete -f deployment.yaml
kubectl delete -f pvc.yaml
Enter fullscreen mode Exit fullscreen mode

Check the PersistentVolume using kubectl get pv - you will see that the STATUS has now changed to Released (from Bound). Confirm that the Azure Disk still exists

az disk list -g $AKS_NODE_RESOURCE_GROUP
Enter fullscreen mode Exit fullscreen mode

It does! And the reason is that we had overridden the default behavior by using a custom Storage Class to set reclaimPolicy to Retain

Once you're done, you can delete the resource group which in turn will delete the AKS cluster and associated Azure Disks

To clean up 

az group delete --name $AZURE_RESOURCE_GROUP --yes --no-wait
Enter fullscreen mode Exit fullscreen mode

That's all for this blog. I really hope you found it useful! 😃 Please like, follow and reach out to me on Twitter for feedback/suggestions or feel free to drop a comment 👇👇

💖 💪 🙅 🚩
abhirockzz
Abhishek Gupta

Posted on September 30, 2019

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

How to use custom Storage Classes in Kubernetes
kubernetes How to use custom Storage Classes in Kubernetes

September 30, 2019

Tutorial: Basics of Kubernetes Volumes (Part 1)
kubernetes Tutorial: Basics of Kubernetes Volumes (Part 1)

September 24, 2019