TheLazy.Dev
Portfolio GitHub

Blog

Devops Project - CI/CD Jenkins Ansible Kubernetes

nithinalias

nithinalias

Posted on April 5, 2022

Devops Project - CI/CD Jenkins Ansible Kubernetes

Kubernetes on AWS





                                               Dockerhub 
                                              /        \
                                            /            \                          
                                      Push/Image       Pull\Image        
                                        /                    \
      Pull Code       Copy Artifacts  /     Deploy Container   \
Github--------->Jenkin-------------->Ansible--------------->Kubernetes         |
                    |  
                    | Build Code
                   \|/
                  Maven
Enter fullscreen mode Exit fullscreen mode

Setup Kubernetes on Amazon EKS

You can follow same procedure in the official AWS document Getting started with Amazon EKS – eksctl

Pre-requisites:

  • an EC2 Instance
  • Install AWSCLI latest verison

Setup kubectl

  • Download kubectl version 1.21
  • Grant execution permissions to kubectl executable
  • Move kubectl onto /usr/local/bin
  • Test that your kubectl installation was successful 
   curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.21.2/2021-07-05/bin/linux/amd64/kubectl
   chmod +x ./kubectl
   mv ./kubectl /usr/local/bin 
   kubectl version --short --client
Enter fullscreen mode Exit fullscreen mode

Setup eksctl

  • Download and extract the latest release
  • Move the extracted binary to /usr/local/bin
  • Test that your eksclt installation was successful 
   curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
   sudo mv /tmp/eksctl /usr/local/bin
   eksctl version
Enter fullscreen mode Exit fullscreen mode

Create an IAM Role and attache it to EC2 instance

Note: create IAM user with programmatic access if your bootstrap system is outside of AWS

IAM user should have access to

IAM

EC2

CloudFormation

Note: Check eksctl documentaiton for Minimum IAM policies

Create your cluster and nodes

   eksctl create cluster --name cluster-name  \
   --region region-name \
   --node-type instance-type \
   --nodes-min 2 \
   --nodes-max 2 \ 
   --zones <AZ-1>,<AZ-2>

   example:
   eksctl create cluster --name valaxy-cluster \
   --region ap-south-1 \
   --node-type t2.small \
Enter fullscreen mode Exit fullscreen mode

To delete the EKS clsuter

   eksctl delete cluster cluster-name --region ap-south-1
Enter fullscreen mode Exit fullscreen mode

Validate your cluster using by creating by checking nodes and by creating a pod

   kubectl get nodes
   kubectl run tomcat --image=tomcat
Enter fullscreen mode Exit fullscreen mode

Deploying Nginx pods on Kubernetes

Deploying Nginx Container

    kubectl create deployment  demo-nginx --image=nginx --replicas=2 --port=80
    # kubectl deployment regapp --image=valaxy/regapp --replicas=2 --port=8080
    kubectl get all
    kubectl get pod
Enter fullscreen mode Exit fullscreen mode

Expose the deployment as service. This will create an ELB in front of those 2 containers and allow us to publicly access them.

   kubectl expose deployment demo-nginx --port=80 --type=LoadBalancer
   # kubectl expose deployment regapp --port=8080 --type=LoadBalancer
   kubectl get services -o wide
Enter fullscreen mode Exit fullscreen mode

Create Pod and Service using manifest file

Create pod manifest file

sudo su -
vim pod.yml

apiVersion: v1
kind: Pod
metadata:
  name: demo-pod
  labels: 
    app: demo-app

spec:
  containers: 
    - name: demo-nginx
      image: nginx
      ports:
        - name: demo-nginx
          containerPort: 80
Enter fullscreen mode Exit fullscreen mode

Create service manifest file

vim service.yml

apiVersion: v1
kind: Service
metadata:
  name: demo-service
  labels: 
    app: demo-app

spec:
  ports:
  - name: nginx-port
    port: 80
    targetPort: 80
  selector:
    app: demo-app
  type: LoadBalancer
Enter fullscreen mode Exit fullscreen mode

Run pod and service

kubectl apply -f pod.yml
kubectl get pod

kubectl apply -f service.yml
kubectl get service

kubectl get all
kubectl get pods -o wide
kubectl get services -o wide
kubectl describe pod podname
kubectl describe service servicename
Enter fullscreen mode Exit fullscreen mode

Setup Pod and Service

                          selector:                labels:
                            app: demo-app            app: demo-app
External network-------------->Service------------------->Pod
Enter fullscreen mode Exit fullscreen mode

As we know there will be large number of pods,When we send a request from External network to Service it looks for selector app: demo-app and forward the requset to the pods having Labels app: demo-app.That is why we use selector in service manifest file and labels in pod manifest file.

Integrating Kubernetes in CI/CD pipeline

Go to ansible machine

create a user and allow password authentication.

sudo su -
adduser ansadmin
id ansadmin

vim /etc/ssh/sshd_config

PasswordAuthentication yes
#PasswordAuthentication no

systemctl restart ssh
Enter fullscreen mode Exit fullscreen mode

Modify visudo for created user.

vim /etc/sudoers

# User privilege specification
root    ALL=(ALL:ALL) ALL
ansadmin ALL=(ALL:ALL) NOPASSWD: ALL
Enter fullscreen mode Exit fullscreen mode

User Privilege Lines

root ALL=(ALL:ALL) ALL The first field indicates the username that the rule will apply to (root).

root ALL=(ALL:ALL) ALL The first “ALL” indicates that this rule applies to all hosts.

root ALL=(ALL:ALL) ALL This “ALL” indicates that the root user can run commands as all users.

root ALL=(ALL:ALL) ALL This “ALL” indicates that the root user can 
run commands as all groups.

root ALL=(ALL:ALL) ALL The last “ALL” indicates these rules apply to all commands.
Enter fullscreen mode Exit fullscreen mode

Generate ssh-key.

su - ansadmin
ssh-keygen
Enter fullscreen mode Exit fullscreen mode

Go to Kubernetes machine

Create Deployment and Service using manifest file

Create deployment manifest file

sudo su -
vim deployment.yml

apiVersion: v1
kind: Deployment
metadata:                        --->Deployment name and label
  name: demo-webapp 
  labels: 
    app: webapp

spec:
  replicas: 2
  selector:                      --->Create 2 pods from pod template
    matchLabels:
       app: webapp

template:                                                    |
    metadata:                    ---> pod definition         |        
      labels:                                                |       
        app: webapp                                          |
                                                             |           
    spec: containers                                         |---> Template to create a pod                                     |
    - name: webapp                                           |
      image: nithinalias/mytomcat ---> container definition  |
      imagePullPolicy: Always                                |
      ports:                                                 |
      - containerPort: 8080                                  |   

strategy:
  type: RollingUpdate
  rollingUpdate:                 --->make sure only one pod updated at a time
    maxSugre: 1
    maxUnavialable: 1
Enter fullscreen mode Exit fullscreen mode

Create service manifest file

vim service.yml

apiVersion: v1
kind: Service                    ---> Resource Type

metadata:
  name: demo-service             ---> service name and label
  labels: 
    app: webapp

spec:
  ports:
  - name: nginx-port
    port: 8080                   ---> port number exposed at cluster level
    targetPort: 8080             ---> port that container listening

  selector:                      ---> To which deployment it can send traffic
    app: webapp
  type: LoadBalancer             ---> service type
Enter fullscreen mode Exit fullscreen mode

labels in deployment manifest file need to match with selector in service manifest file.containerPort in deployment manifest file need to match with targetPort in service manifest file.

sudo su -
passwd
adduser ansadmin
id ansadmin
docker login
Enter fullscreen mode Exit fullscreen mode

Allow password authentication.

vim /etc/ssh/sshd_config

PermitRootLogin yes
PasswordAuthentication yes
#PasswordAuthentication no

systemctl restart ssh
Enter fullscreen mode Exit fullscreen mode

Modify visudo for created user.

vim /etc/sudoers

# User privilege specification
root    ALL=(ALL:ALL) ALL
ansadmin ALL=(ALL:ALL) NOPASSWD: ALL
Enter fullscreen mode Exit fullscreen mode

Go to ansible machine and add ip address of kubernetes machine and ansible machine(then only jenkin server can access ansible and kubernetes machine) inside ansible host file.

vim /etc/ansible/hosts

[ansible]
192.168.33.25
[kubernetes]
192.168.33.30
Enter fullscreen mode Exit fullscreen mode

copy the ssh public key to kubernetes machine and ansible machine itself(localhost).

su - ansadmin
ssh-copy-id ansadmin@ip-of-ansiblemachine
ssh-copy-id ansadmin@ip-of-kubernetesmachine
ssh-copy-id root@ip-of-kubernetesmachine
ansible all -m ping
Enter fullscreen mode Exit fullscreen mode

Now authorized key will be generated inside /home/ansadmin/.ssh/authorized_keys of kubernetes,ansible machine and /root/.ssh/authorized_keys of kubernetes machine.

create a directory,give ownership of ansadmin,add ansadmin to docker group and docker.sock permissions to all users.

sudo su -
mkdir /opt/docker
chown ansadmin:ansadmin /opt/docker
usermod -aG docker ansadmin
id ansadmin
chmod 777 /var/run/docker.sock
systemctl restart docker
su - ansadmin
docker login
Enter fullscreen mode Exit fullscreen mode

Create dockerfile for tomcat image creation.

vim /opt/docker/dockerfile

FROM tomcat
RUN cp -R /usr/local/tomcat/webapps.dist/* /usr/local/tomcat/webapps
COPY ./*.war /usr/local/tomcat/webapps
Enter fullscreen mode Exit fullscreen mode

Generate ansible-playbook for building tomcat image.

vim /opt/docker/build_tomcatimage.yml

---

- hosts: ansible

  tasks:
  - name: create docker image
    command: docker build -t mytomcat .
    args:
     chdir: /opt/docker

  - name: create tag to push image onto dockerhub
    command: docker tag mytomcat nithinalias/mytomcat

  - name: push docker image
    command: docker push nithinalias/mytomcat
Enter fullscreen mode Exit fullscreen mode

Generate ansible-playbook to run deployment and service manifest file.

vim /opt/docker/run_kube_manifestfile.yml

---

- hosts: kubernetes
  user: root

  tasks:
  - name: run deployment manifestfile on kubernetes
    command: kubectl apply -f deployment.yml

  - name: run service manifestfile on kubernetes
    command: kubectl apply -f service.yml

  - name: update deployment with new pods if image updated in dockerhub
    command: kubectl rollout restart deployment.apps/demo-webapp

# we don't want to mention exact path like /root/deployment.yml or /root/service.yml, since we are performing the task as root user.
Enter fullscreen mode Exit fullscreen mode

Integrate Ansible with Jenkins

we need to add ansadmin ssh connection of ansible machine.

Manage Jenkins - Configure System - SSH servers - Add - Add name(ansible-server),Hostname(ip-of-ansible machine),username
(ansadmin) - Advanced - Enable Use password authentication or use a different key - Add password of user ansadmin(instead you can use ssh-key/path of ssh-key if present) - Test configuration - Apply - save.

Continuous Integration using Jenkins

New Item - Add item name(CI webapp),select Maven project - OK - Add description - Select Git and add Repositories URL,Branch - Build Triggers(Poll SCM=* * * * * means every minute check the repository and if there any update it will trigger build - Build periodically means it will trigger build periodically even if repository not updated) - Build(Add Root POM=pom.xml,Goals and options=clean install) - Add post-build Action - Send build artifacts over SSH - Select SSH server name(ansible-server),Add sourcefile(webapp/target/*.war),Add Remove prefix(webapp/target),Add Remote directory="//opt//docker"(if it is blank webapp.war will be in /home/ansadmin),
Exec command ="ansible-playbook /opt/docker/build_tomcatimage.yml" - Add post-build action - Build other projects - Go to post-build action(top) - Add Projects to build="CD webapp" - Enable Trigger only if build is stable - Apply - Save - Build Now - Console output.You can see all build outcomes in /var/lib/jenkins/workspace/CI webapp/webapp/target.surefire contain the reports of build and webapp.war is called artifacts.This webapp.war will be in ansible machine /opt/docker directory and copy this to tomcat image.The image created inside ansible machine is pushed to dockerhub.

Continuous Deployment using Jenkins

New Item - Add item name(CD webapp) - OK - Add description - Add post-build Action - Send build artifacts over SSH - Select SSH server name(ansible-server),Exec command =
"ansible-playbook /opt/docker/run_kube_manifestfile.yml" - Apply - Save.

Go to Kubernetes machine

kubectl get service
Enter fullscreen mode Exit fullscreen mode

copy the EXTERNAL-IP of your service and paste it in browser.Now you get the webapp.
OR

You can go to LoadBalancer in AWS - Description - Copy DNS name - paste it in browser.Now you get the webapp.

Source: https://github.com/yankils

💖 💪 🙅 🚩
nithinalias
nithinalias

Posted on April 5, 2022

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

What was your win this week?
weeklyretro What was your win this week?

November 29, 2024

How to Use KitOps with MLflow
beginners How to Use KitOps with MLflow

November 29, 2024

Caught in the Crunch My Journey from Snacks to 2 Million Exposed Users Privacy
privacy Caught in the Crunch My Journey from Snacks to 2 Million Exposed Users Privacy

November 30, 2024

Submission for the DevCycle Feature Flag Challenge: Feature Flag Funhouse
devchallenge Submission for the DevCycle Feature Flag Challenge: Feature Flag Funhouse

November 30, 2024

How to Detect and Defend Against SQL Injection Attacks - Part 2
security How to Detect and Defend Against SQL Injection Attacks - Part 2

November 30, 2024