Jenkins - Auto-build your dockerised local Environment on code commit

Previously, we set up a local JS and Python environment using docker and docker-compose. In this post, we will automate the build process in Docker using Jenkins. This will allow us focus more on our code while developing.

Please read the first part of this tutorial here if you haven't.

Let's get started!

What is Jenkins?

Jenkins is an open-source automation server. It's used to support building, deploying, and automating any project. Jenkins is a continuous integration and delivery (CI/CD) tool that is widely used interchangeably. Jenkins's capacity to divide tasks among several nodes is one of its most powerful features.

Setting up Jenkins

To follow along, clone the repository using the following commands.
git clone https://github.com/jagkt/local_dev_env.git
cd local_dev_env
Now you should have the following project structure to start with:
.
├── node
│ ├── index.js
│ └── package.json
└── py
│ ├── Dockerfile
│ ├── requirements.txt
│ └── main.py
├── LICENSE
├── Makefile
├── README.md
├── Jenkinsfile
└── docker-compose.yml

Here, Jenkinsfile is introduced to the project directory, this is where we will declare our declarative pipeline script telling Jenkins how to go about the build process. Also, the Jenkins service is added in the docker-compose.yml file with the following code.

jenkins:
    image: jenkins/jenkins:lts
    privileged: true
    user: root
    ports:
      - 8080:8080
      - 50000:50000
    container_name: jenkins
    volumes:
      - /home/${whoami}/jenkins:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/bin/docker:/usr/bin/docker

Here, the Jenkins service is defined by pulling the latest Jenkins image with root privileges. The container is run with host networking, and we tell Docker to redirect ports 8080 and 50000 to the host’s network. Also, we gave it a container name called Jenkins.

Finally, the /home/${whoami}/Jenkins (which will contain the Jenkins configuration) is mapped to /var/jenkins_home in the container. Ensure to create the Jenkins configuration directory on your host and change /home/${whoami}/ to your user’s home directory or the path you created the new directory. Also, the /var/run/docker.sock and /usr/bin/docker on the host are mapped to the /var/run/docker.sock and /usr/bin/docker respectively. This is to allow the Jenkins container to have access to the host docker daemon.

Now run the Jenkins service

Run docker-compose in the directory where you placed docker-compose.yml file.
$ docker-compose up -d jenkins

Allow Jenkins to use docker-compose commands

Because we will be using the docker-compose command in our Jenkins pipeline script, we need to ensure Jenkins can run the commands.

On your host, exec into the running Jenkins container.

docker exec -it jenkins /bin/bash

run the following commands inside the Jenkins container.

groupadd -g 997 docker
gpasswd -a jenkins docker
curl -L https://github.com/docker/compose/releases/tag/1.29.2/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose

#confirm the docker-compose is installed
docker-compose version

the last command output should be similar to this.

Jenkins Configuration

Now point a web browser at port 8080 on your host system and use the actual IP address or domain name for the server you are using Jenkins on. In our case http://localhost:8080. A page opens prompting you to Unlock Jenkins. Obtain the required administrator password in the next step.

Obtain the default Jenkins unlock password by opening the terminal and running the following command:

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

The system returns an alphanumeric code. Enter that code in the Administrator password field and click Continue.
The setup prompts to either install suggested plugins or Select plugins to install. It’s fine to simply install the suggested plugins.

The next step is the Create First Admin User. Enter the credentials you want to use for your Jenkins administrator, then click Save and Continue. Here, Jenkins user is used.

click “Save and Continue”
click “Start using Jenkins”

Create a Jenkins project

we need to create a pipeline that builds our images automatically. From the Jenkins dashboard:

select "New Item", on the pop-up window, enter the project name in the "Item Name" textboard, then select Pipeline and click "OK"

On the project's configuration page, under the "General" tab, scroll down to "Build trigger" and select "GitHub hook trigger for GITScm polling"

Scroll down to the Pipeline section, under "Definition", select "Pipeline script from SCM", and under SCM, select "Git".
fill in the "repository URL" and add your GitHub credential (you can generate a token on GitHub for authentication)

On the branch to build from, specify the branch name (in this case, the main branch).

Scroll to the script path and ensure Jenkinsfile is specified.

Finally, select "Manage Jenkins" on your dashboard and select "Plugins". Under the "Available Plugins", search for "Docker Pipeline" and install.

Jenkinsfile

The Jenkinsfile we added to the project directory contains the below declarative pipeline script.

pipeline {

    agent any

    stages {
        stage('Checkout Source') {
            steps {
                git branch: 'main ', url: 'https://github.com/jagkt/local_dev_env.git'
            }
        }

        stage('Docker Compose Build image') {
            steps {
                    sh "docker-compose build"
                    sh "docker-compose up -d py_app"
                    sh "docker-compose up -d node_app"
            }
        }
    } 
}

Here,

pipeline is Declarative Pipeline-specific syntax that defines a "block" containing all content and instructions for executing the entire Pipeline.
agent is Declarative Pipeline-specific syntax that instructs Jenkins to allocate an executor (on a node) and workspace for the entire Pipeline. Here, any agent is specified.
stage is a syntax block that describes a stage of this Pipeline. stage blocks are optional in Scripted Pipeline syntax. Here we have two stages build process: checkout source and image build process
steps is Declarative Pipeline-specific syntax that describes the steps to be run in this stage.
sh is a Pipeline step (provided by the Pipeline: Nodes and Processes plugin) that executes the given shell command. First, We build our images before running their containers.

Read more about Jenkins' declarative pipeline here. Also, ensure to set the checkout source URL to your GitHub project repository and specify which branch to build from.

Automatically Run Build on Code Commit

First, we need to get the hook URL of Jenkins. From the Dashboard, click "Manage Jenkins", then click "System"
Scroll down to the "Github" section, click "Advance" under the " Github server", and check the box for "Specify another hook URL for GitHub configuration". Click "Save"

As shown above, the URL is http://localhost:8080/github-webhook/. This will be used to add a webhook to the GitHub project repository. The challenge here is, how does GitHub determine the host where our Jenkins container is running?
A quick way to solve the challenge above is to make the Jenkins service accessible in the public domain. I recently wrote an article on how to quickly set this up using Localrun.

Read the article here

Obtain the domain given to you after running Localrun on your host. say for example we have https://2d70a750dc2554.lhr.life.
Go to your project repository on GitHub, click "Settings", click "Webhooks", and click "Add webhook".
Configure as shown below and click "Add Webhook".

Here, we have replaced http://localhost:8080/ with https://2d70a750dc2554.lhr.life/

Note: localrun domain is active for a few minutes. This means you need to update the webhook payload URL every time with the new domain to continue the auto-build process. Alternatively, you can use other services that provide longer time usage.

Now that the setup is completed, Jenkins will now run the build whenever you push/commit code to the main branch of your GitHub repository.

On your Dashboard, you can see your created pipeline, click on it, and you can see your build history or manually run your build.

Conclusion

I hope you now have a solid idea of setting up Jenkins to automate your local environment build process. In summary, we observed the following:

Created a Jenkins service and allowed it to access the host docker daemon.
configured Jenkins pipeline job to auto-build whenever we commit code changes to the GitHub repository.

If you experience difficulty while going through this tutorial, you can drop the challenge(s) faced in the comment section.

Blog