TheLazy.Dev
Portfolio GitHub

Blog

beginners |tutorial |eks |kubernetes

Deploy a python app (Docker Image) to Kubernetes cluster via AWS EKS

jasper475

Jasper Rodda

Posted on September 20, 2023

Deploy a python app (Docker Image) to Kubernetes cluster via AWS EKS

This post will help you to deploy a python game app into a highly available cluster via AWS EKS (a Managed control plane service from AWS).

Pre-requisites

  1. Install AWS CLI
  2. Install Kubectl
  3. Install eksctl
  4. Install Heml

Step 1 : Configure AWS root account 

PS C:\WINDOWS\system32> AWS configure
AWS Access Key ID [****************FINA]:
AWS Secret Access Key [****************R6U5]:
Default region name [us-east-1]:
Default output format [json]:
PS C:\WINDOWS\system32>
Enter fullscreen mode Exit fullscreen mode

Step 2 : Create EKS cluster

  • Create EKS cluster via CLI eksctl create cluster --name demo-cluster --region us-east-1 --fargate
  • Output: It should take ~5 to 15 mins to create cluster in AWS 
PS C:\WINDOWS\system32> eksctl create cluster --name demo-cluster --region us-east-1 --fargate
2023-09-19 23:43:58 [ℹ]  eksctl version 0.157.0
2023-09-19 23:43:58 [ℹ]  using region us-east-1
2023-09-19 23:43:58 [ℹ]  setting availability zones to [us-east-1a us-east-1d]
2023-09-19 23:43:58 [ℹ]  subnets for us-east-1a - public:192.168.0.0/19 private:192.168.64.0/19
2023-09-19 23:43:58 [ℹ]  subnets for us-east-1d - public:192.168.32.0/19 private:192.168.96.0/19
2023-09-19 23:43:58 [ℹ]  using Kubernetes version 1.25
2023-09-19 23:43:58 [ℹ]  creating EKS cluster "demo-cluster" in "us-east-1" region with Fargate profile
2023-09-19 23:43:58 [ℹ]  if you encounter any issues, check CloudFormation console or try 'eksctl utils describe-stacks --region=us-east-1 --cluster=demo-cluster'
2023-09-19 23:43:58 [ℹ]  Kubernetes API endpoint access will use default of {publicAccess=true, privateAccess=false} for cluster "demo-cluster" in "us-east-1"
2023-09-19 23:43:58 [ℹ]  CloudWatch logging will not be enabled for cluster "demo-cluster" in "us-east-1"
2023-09-19 23:43:58 [ℹ]  you can enable it with 'eksctl utils update-cluster-logging --enable-types={SPECIFY-YOUR-LOG-TYPES-HERE (e.g. all)} --region=us-east-1 --cluster=demo-cluster'
2023-09-19 23:43:58 [ℹ]
2 sequential tasks: { create cluster control plane "demo-cluster",
    2 sequential sub-tasks: {
        wait for control plane to become ready,
        create fargate profiles,
    }
}
2023-09-19 23:43:58 [ℹ]  building cluster stack "eksctl-demo-cluster-cluster"
2023-09-19 23:43:59 [ℹ]  deploying stack "eksctl-demo-cluster-cluster"
2023-09-19 23:44:29 [ℹ]  waiting for CloudFormation stack "eksctl-demo-cluster-cluster"
2023-09-19 23:45:00 [ℹ]  waiting for CloudFormation stack "eksctl-demo-cluster-cluster"
2023-09-19 23:46:00 [ℹ]  waiting for CloudFormation stack "eksctl-demo-cluster-cluster"
Enter fullscreen mode Exit fullscreen mode
  • Output : AWS console

Image description

  • Download Kubeconfig aws eks update-kubeconfig --name demo-cluster --region us-east-1
  • output: 
PS C:\WINDOWS\system32> aws eks update-kubeconfig --name demo-cluster --region us-east-1
Added new context arn:aws:eks:us-east-1:21344354364:cluster/demo-cluster to C:\Users\Jasper\.kube\config
Enter fullscreen mode Exit fullscreen mode

Step 3 : Create custom fargate-profile

  • To Create custom fargate-profile use below snippet 
$ eksctl create fargateprofile \
    --cluster demo-cluster \
    --region us-east-1 \
    --name alb-sample-app \
    --namespace game-2048
2023-09-20 00:17:40 [ℹ]  creating Fargate profile "alb-sample-app" on EKS cluster "demo-cluster"
2023-09-20 00:19:50 [ℹ]  created Fargate profile "alb-sample-app" on EKS cluster "demo-cluster"
- output : 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ncyy3edj3wkt7ekb48s4.jpg)
Enter fullscreen mode Exit fullscreen mode

Step 4 : Deploy using deployment.yml file

  • Deploy via deployment.yaml , service and Ingress (Note: Please attach Ingress controller to this to create a ALB for traffic to flow through) 
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.5.4/docs/examples/2048/2048_full.yaml
namespace/game-2048 created
deployment.apps/deployment-2048 created
service/service-2048 created
ingress.networking.k8s.io/ingress-2048 created
Enter fullscreen mode Exit fullscreen mode
  • Get pods kubectl get pods -n game-2048 
$ kubectl get pods -n game-2048
NAME                               READY   STATUS    RESTARTS   AGE
deployment-2048-5686bb4958-22w8c   1/1     Running   0          7m24s
deployment-2048-5686bb4958-2jvnh   1/1     Running   0          7m25s
deployment-2048-5686bb4958-94dgg   1/1     Running   0          7m25s
deployment-2048-5686bb4958-fqxmg   1/1     Running   0          7m24s
deployment-2048-5686bb4958-lndww   1/1     Running   0          7m25s
Enter fullscreen mode Exit fullscreen mode
  • search for service $ kubectl get svc -n game-2048
  • Note: The service has CLUSTER_IP with NodePort but no EXTERNAL-IP: ie., Anyone that has access to VPC can talk to the pod using nodeIP address followed by name of port 10.100.219.219:80:30929 
$ kubectl get svc -n game-2048
NAME           TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
service-2048   NodePort   10.100.219.219   <none>        80:30929/TCP   10m
Enter fullscreen mode Exit fullscreen mode
  • Get Ingress : We use this ingress to make someone outside the AWS VPC access the pod. kubectl get ingress -n game-2048
  • Output: Note: There is no Address associated to ingress. There has to be an ingress controller. once we deploy ingress controller 
$ kubectl get ingress -n game-2048
NAME           CLASS   HOSTS   ADDRESS   PORTS   AGE
ingress-2048   alb     *                 80      16m
Enter fullscreen mode Exit fullscreen mode

Step 5 : Pre-requisite to Step #6 (create ALB Ingress-Controller )

  • This will read from .yaml file and it will read 'ingress-2048' and configure ALB - example: target group ## Pre-requisite: Configure/Associate OIDC connector
  • we need IAM OIDC connector because the ALB controller (K8s pod) needs to access Application load balancer (ALB)
  • to talk to ALB it needs IAM OIDC provider. 
eksctl utils associate-iam-oidc-provider --cluster demo-cluster --approve
2023-09-20 01:02:21 [ℹ]  will create IAM Open ID Connect provider for cluster "demo-cluster" in "us-east-1"
2023-09-20 01:02:22 [✔]  created IAM Open ID Connect provider for cluster "demo-cluster" in "us-east-1"
Enter fullscreen mode Exit fullscreen mode
  • Install ALB controller (Its a K8s pod ): It needs the following
  • Architecture flow: ALB controller --> (K8s Pod) --> Need access to AWS services such as ALB
  1. Download IAM policy curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.5.4/docs/install/iam_policy.json
  2. Create IAM Policy for Pod to access AWS Service/ALB
  3. Create IAM Role for Pod : an EKS LB Controller Role and attach this role to service account to Pod.
  • Output 1 : 
$ curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.5.4/docs/install/iam_policy.json
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  8386  100  8386    0     0   5093      0  0:00:01  0:00:01 --:--:--  5144
Enter fullscreen mode Exit fullscreen mode
  • Output 2 : 
$ aws iam create-policy \
    --policy-name AWSLoadBalancerControllerIAMPolicy \
    --policy-document file://iam_policy.json
{
    "Policy": {
        "PolicyName": "AWSLoadBalancerControllerIAMPolicy",
        "PolicyId": "ASADF23423SDFG",
        "Arn": "arn:aws:iam::123435234:policy/AWSLoadBalancerControllerIAMPolicy",
        "Path": "/",
        "DefaultVersionId": "v1",
        "AttachmentCount": 0,
        "PermissionsBoundaryUsageCount": 0,
        "IsAttachable": true,
        "CreateDate": "2023-09-20T06:13:14+00:00",
        "UpdateDate": "2023-09-20T06:13:14+00:00"
    }
}
Enter fullscreen mode Exit fullscreen mode
  • Output 3: eksctl create iamserviceaccount --cluster=demo-cluster --namespace=kube-system --name=aws-load-balancer-controller --role-name AmazonEKSLoadBalancerControllerRole --attach-policy-arn=arn:aws:iam::342523452346:p olicy/AWSLoadBalancerControllerIAMPolicy --approve 
$ eksctl create iamserviceaccount   --cluster=demo-cluster   --namespace=kube-system   --name=aws-load-balancer-controller   --role-name AmazonEKSLoadBalancerControllerRole   --attach-policy-arn=arn:aws:iam::342523452346:p
olicy/AWSLoadBalancerControllerIAMPolicy   --approve
2023-09-20 01:19:19 [ℹ]  1 iamserviceaccount (kube-system/aws-load-balancer-controller) was included (based on the include/exclude rules)
2023-09-20 01:19:19 [!]  serviceaccounts that exist in Kubernetes will be excluded, use --override-existing-serviceaccounts to override
2023-09-20 01:19:19 [ℹ]  1 task: { 
    2 sequential sub-tasks: {
        create IAM role for serviceaccount "kube-system/aws-load-balancer-controller",
        create serviceaccount "kube-system/aws-load-balancer-controller",
    } }2023-09-20 01:19:19 [ℹ]  building iamserviceaccount stack "eksctl-demo-cluster-addon-iamserviceaccount-kube-system-aws-load-balancer-controller"
2023-09-20 01:19:19 [ℹ]  deploying stack "eksctl-demo-cluster-addon-iamserviceaccount-kube-system-aws-load-balancer-controller"
2023-09-20 01:19:19 [ℹ]  waiting for CloudFormation stack "eksctl-demo-cluster-addon-iamserviceaccount-kube-system-aws-load-balancer-controller"
2023-09-20 01:19:49 [ℹ]  waiting for CloudFormation stack "eksctl-demo-cluster-addon-iamserviceaccount-kube-system-aws-load-balancer-controller"
2023-09-20 01:19:50 [ℹ]  created serviceaccount "kube-system/aws-load-balancer-controller"
Enter fullscreen mode Exit fullscreen mode

Step 6 : Create ALB Ingress-Controller via helm chart

  • Add helm repo helm repo add eks https://aws.github.io/eks-charts 
$ helm repo add eks https://aws.github.io/eks-charts
"eks" has been added to your repositories
Enter fullscreen mode Exit fullscreen mode
  • Get VPC ID = vpc-0b8ke79b820c877c0, Region= 'us-east-2' 
$ helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=demo-cluster \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller \
  --set region=us-east-2 \
  --set vpcId=vpc-0b8ke79b820c877c0
NAME: aws-load-balancer-controller
LAST DEPLOYED: Wed Sep 20 01:32:34 2023
NAMESPACE: kube-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
AWS Load Balancer controller installed!
Enter fullscreen mode Exit fullscreen mode
  • Verify Load balancer is created and at least 2 replicas of it. kubectl get deployment -n kube-system aws-load-balancer-controller
  • output : 
$ kubectl get deployment -n kube-system aws-load-balancer-controller
NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
aws-load-balancer-controller   2/2     2            2           114s
Enter fullscreen mode Exit fullscreen mode
  • Get deployment : available repicas is 2/2 
$ kubectl get deploy -n kube-system
NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
aws-load-balancer-controller   2/2     2            2           5m18s
coredns                        2/2     2            2           108m
Enter fullscreen mode Exit fullscreen mode

Step 7: Verify if "ALB controller - K8s Pod " created ALB

  • Verify if "ALB controller - K8s Pod created via ingress resources" created ALB or not in AWS console
  • AWS console --> EC2 --> ALB
  • Output: Image description 
$ kubectl get ingress -n game-2048
NAME           CLASS   HOSTS   ADDRESS                                                                  PORTS   AGE
ingress-2048   alb     *       k8s-game2048-ingress2-bacc1c6c73-234532456.us-east-1.elb.amazonaws.com   80      76m
Enter fullscreen mode Exit fullscreen mode

Step 8 : Verify Load balancer DNS/ IP in browser

-Output: Congratulations. I was able to deploy python app which is accessible via Load balancer.
Image description

Note:

Usually, DevOps engineers use deployment.yml, service.yaml, ingress.yaml & ingress controller (One time responsibility: for EKS we have to create service account --> Then attach servicde account with IAM role) with On-prem: One can assign ingress controller service account with proper RBAC

Credits:

💖 💪 🙅 🚩
jasper475
Jasper Rodda

Posted on September 20, 2023

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Deploy a python app (Docker Image) to Kubernetes cluster via AWS EKS
beginners Deploy a python app (Docker Image) to Kubernetes cluster via AWS EKS

September 20, 2023