Django on K8s (Part VI: Advanced Topics)
Mohamed M El-Kalioby
Posted on January 13, 2022
This part will be a collection of advanced topics that we need to make use of Kubernetes and it is the last part of this tutorial which I hope simplified the technology and gave you a good start, so let's start
For the sake of simplicity, we will use the SQLite version (master
branch) of the application, but the MySQL branch will work as well.
Readiness and Liveness checks
They are similar in terms of configuration, but different when each gets invoked, that is why most guides state them together.
Rediness Check: When true the container is ready to accept traffic, can be useful if the container needs some time to prepare the environment and start receiving traffic.
Liveness Check: It is a check that Kubernetes will run periodically to make sure that the container is alive and kicking, in case it fails, the container will be considered dead and it will be terminated and a new container will be started.
Note: We said 'containers' and not 'pods' are the checks are per container.
Types of the Checks
The checks can be one of the following
- command: that returns 0 if succeed, error otherwise
- HttpGet: which is a success if the code is between 200 and 399.
- tcpSocket: if Kubernetes can open the connection to the TCP port then it is a success else a failure.
Configure Probes:
As stated in the documentation, there are several configurations for the probes
- initialDelaySeconds: the number of seconds before the check is initiated (after container starts), default is 0
- periodSeconds: The interval between the checks, default 10 seconds
- failureThreadhold: Try this number of times before considering the probe failed (which means restarting the container), default to 3
Example
Note: In our case (web-app), both readiness and liveness checks shall be 'httpGET', but the sake of the tutorial, we will configure the rediness check as a command and liveness to be httpGET
in the web-app container section, add the following YAML which is available in django-example_checks.yaml
containers:
- name: web-app
image: django-example:v1.0
ports:
- containerPort: 80
readinessProbe: #<-- Start here
periodSeconds: 5
initialDelaySeconds: 5
failureThreshold: 100
exec:
command:
- cat
- /tmp/ready
livenessProbe:
httpGet:
path: /
port: 80
initialDelaySeconds: 2
periodSeconds: 3 #<-- End here
volumeMounts:
- name: sqlite-volume
mountPath: /app/db.sqlite3
So the container are ready, if '/tmp/ready' exists, the container is alive it can response to the HTTP request on port 80.
Note: For the sake of the demo, we set the failureThreshold very high that should NOT be the case on a real environment.
Testing it
django-on-k8s$ kubectl apply -f django-example_checks.yaml
Wait for few seconds and then run
kubectl get po -l app=django-example
You will see that the po is not ready, so to know why it is ready run describe as shown
kubectl describe po -l app=django-example
and check at the end. You will see that the pod is unheathly as rediness probe is failing as shown
To solve the issue, connect to the pod to touch the final, this is describe in the next section 'Executing Commands in Pods'
Executing Commands in Pods
Sometimes, we need to run commands in the containers like we need to run a management command, here if it we demo, how to connect to the pod we created and touch the '/tmp/ready' so the the container becomes ready.
kubectl exec django-example-5c79c94df6-9vkr7 -c web-app -it -- /bin/bash
Notes
1. 'django-example-5c79c94df6-9vkr7' is the pod name
2. -c takes a container name, useful if there are multiple containers in the pod
3. What is after '--' is the command to pass to container, so here we started a bash session
4. '-it' is a flag to state that this is an interactive session.
Then once you get the terminal to the container, enter the following command
touch /tmp/ready
Then press CTRL+D to exit, and after check the status of the pod and you will find it ready as shown below.
Rolling Updates
Writing a complete bug-free application doesn't exist so we need to make an update to our web application now.
Rather greeting the user by the username, we will greet him by the first name.
Note: You can move to update
branch if you dont want to make the change manually.
To do the update open 'django_app/accounts/templates/home.html' and Go to line 18 and change it to
<h1>Welcome {{ request.user.first_name }}!</h1>
Next, you need to build a new docker image, to save time, build it against 'minikube docker'
django-on-k8s$ eval $(minikube docker-env)
django-on-k8s$ docker build -t django-example:v1.1 .
Then edit the deployment on Kubernetes by running
kubectl edit deployments.apps django-example
In the file, search for the image and change it to the new image and also, remove the readiness check as shown below
Now run
kubectl get po
See the old pod are terminating and a new one is starting.
Lets try the new application
Oops!, we made an issue, we didn't handle the case when there is no first name, like with the account created by 'create superuser', so first let's rollback the production to the old image, till we fix the issue.
Rollback
When we have an issue like this, don't freak-out, just ask Kubernetes to rollback like this.
First, let's get the history
kubectl rollout history deployment django-example
This will show the change history as shown below
Note: I have multiple changes as I didn't remove the readiness check in the 2nd change.
So we want to go to 1 so we run
kubectl rollout undo deployment django-example --to-revision=1
Why is it ready, because of the ready state, so let's touch the file as we did above.
Let's refresh the website.
Wrap-Up
So in the last part of this tutorial, we showed readiness and liveness probes, executing commands in the container and finally how to roll and rollback an update.
There are some interesting topics we didn't cover, like
ConfigMaps and secrets, which allow to store information in Kubernetes (like the mysql password) instead on saving them on file, also Ingress and Ingress Controllers which may be configured to give Services externally-reachable URLs, load balance traffic, terminate SSL / TLS, and offer name-based virtual hosting.
Finally, I hope this tutorial was useful and you learnt something.
Thank you.
Posted on January 13, 2022
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.