Learn what Docker Compose is, how it makes the life of an Engineer easier and possible use cases

Link to video: https://youtu.be/yFvO8Atszl8

What the heck is Docker Compose and how does it make the life of an Engineer like you easier?

This tutorial will answer that (and more), as simply as possible.

Introduction

Agenda for this tutorial:

• What is Docker Compose?
• Why it exists? How does it differ from docker run?
• How to use Docker Compose?
• When to use it? - use cases

Absolutely new to Docker? Check out: What is Docker?

Want to learn more about Docker Networking? Check out: Docker Networking Summary.

Alright…

What is Docker Compose?

Docker Compose is a tool for defining and running multi-container applications.

With Compose, you use a YAML file to configure your application’s services (containers). Then, with a single command, you can build, start or delete your application services.

Why Docker Compose exists? How does it differ from docker run?

Running multiple containers is a very common scenario.

Take for example a WordPress (WP) application. It consists of a WordPress service that talks to a MySQL database.

We could run both containers using two docker run commands with a bunch of cli arguments. The db container might be started like this:

docker run -d \
  --name db \
  --restart always \
  -v db_data:/var/lib/mysql \
  -e MYSQL_ROOT_PASSWORD=supersecret \
  -e MYSQL_DATABASE=exampledb \
  -e MYSQL_USER=exampleuser \
  -e MYSQL_PASSWORD=examplepass \
  mysql:5.7

Additionally, we might need to isolate these containers from the host or other containerized applications. We could create or remove networks with docker network commands, and modify docker run to take the network as an argument.

Typing out these verbose commands might be fine once or twice. But as the number of containers and configurations grows, they become increasingly harder to manage.

With Compose, we simply define the application’s configuration on a YAML file (named docker-compose.yml by default) like this:

version: '3.9'

services:
  db:
    image: mysql:5.7
    restart: always
    environment:
      MYSQL_DATABASE: exampledb
      MYSQL_USER: exampleuser
      MYSQL_PASSWORD: examplepass
      MYSQL_ROOT_PASSWORD: supersecret
    volumes:
      - db_data:/var/lib/mysql
  wordpress:
    image: wordpress
    restart: always
    ports:
      - 8080:80
    environment:
      WORDPRESS_DB_HOST: db
      WORDPRESS_DB_USER: exampleuser
      WORDPRESS_DB_PASSWORD: examplepass
      WORDPRESS_DB_NAME: exampledb
    volumes:
      - wordpress_data:/var/www/html

volumes:
  wordpress_data:
  db_data: 

This file defines 2 services, db and wordpress. It also specifies the configuration options for each - like the image, environment variables, published ports, volumes, etc.

NOTE: if you don’t understand all these options yet, that’s fine.

After creating this file, we execute docker compose up, and Docker builds and runs our entire application in a new isolated environment (bridge network by default).

Similarly, we can use the docker compose down command to tear everything down (except volumes).

Easy! Right? 👌

This is how compose simplifies running multi-container applications on a single host.

Being able to declare and reuse the configuration as a file makes the life of an Engineer much easier. It also allows us to version control, run tests and review our configuration just like the application’s source code.

Besides the benefits above, Compose provides the following key features:

• Have multiple isolated environments on a single host
• Preserve volume data when containers are created
• Only recreate containers that have changed
• Share variables or configurations between environments

That’s all good to know. But let’s learn…

How to use Docker Compose?

Source code for this demo: https://github.com/AluBhorta/docker-compose-demo

Step 1: Install Docker & Docker Compose

First of all, make sure you have installed:

• Docker
• Docker Compose

NOTE: since Compose version 2, we use docker compose command instead of docker-compose. This tutorial uses version 2.

Step 2: Create a sample web application

Open up a terminal, create a new directory and switch into it:

mkdir docker-compose-demo
cd docker-compose-demo

Add the code for a simple Python web app on a file named app.py:

import time

import redis
from flask import Flask

app = Flask(__name__)
cache = redis.Redis(host='redis', port=6379)

@app.route('/')
def hello():
    count = cache.incr('hits')
    return 'Hello World! I have been seen {} times.\n'.format(count)

This creates a Flask app with a single HTTP endpoint (/). This endpoint returns how many times it has been visited. The count is stored and incremented as an integer with a key named hits in a Redis host named redis.

Then we add the Python dependencies to a requirements.txt file:

flask
redis

After that, we create a Dockerfile - to create a Docker image based on this application:

FROM python:3.7-alpine

WORKDIR /code

RUN apk add --no-cache gcc musl-dev linux-headers

COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt

EXPOSE 5000

COPY . .

ENV FLASK_APP=app.py
ENV FLASK_RUN_HOST=0.0.0.0

CMD ["flask", "run"]

This tells Docker to:

• Build an image starting with the Python 3.7 Alpine Linux image
• Set the working directory to /code
• Install gcc and other dependencies with the apk package manager
• Copy requirements.txt from host to image
• Install the Python dependencies with pip
• Add metadata to the image to describe that the container is listening on port 5000
• Copy the current directory . in the project to the workdir . in the image
• Set environment variables used by the flask command
• Set the default command for the container to flask run

Then we create a docker-compose.yml file for us to use Docker Compose:

version: "3.9"

services:
  redis:
    image: "redis:alpine"
  web:
    build: .
    ports:
      - "8000:5000"
    depends_on:
      - redis

This Compose file defines two services: web and redis.

The redis service uses a public redis:alpine image pulled from the Docker Hub registry.

The web service uses an image that’s built from the Dockerfile in the current directory (.). It then maps port 8000 on the host to port 5000 on the container where the flask server will be running. It also specifies that web depends on redis so that Docker knows to start redis before web.

NOTE: since Compose creates a new bridge network on project startup, web can reach redis simply by using the service’s name ie. redis.

Step 3: Run and test the application

To run the application, all we have to do now is:

docker compose up

Docker will automatically pull the redis image, build our web image and start the containers.

Once deployed, we should now be able to reach the application at localhost:8000 on your browser.

Or alternatively, use curl on a separate terminal to reach the flask application like so:

curl localhost:8000

You should see something like this:

Hello World! I have been seen 1 times.

The count is incremented every time we make a request.

Awesome! 👌

We can list the containers of the Compose project with:

docker compose ps

NOTE: docker compose up will by default attach to your terminal and print the logs from the services. We can use ctrl+c to detach that terminal, but it will stop the services.

To run the services in the background, use -d the flag:

docker compose up -d

If you want to view the logs, use:

docker compose logs -f

NOTE: -f will follow the log output as new logs are generated.

Step 4: Modify the application

Changes are inevitable.

Let us make a change to our app.

The web container is printing out a warning:

WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.

In production, we definitely want to run a production-grade server like gunicorn instead of the development server we get with flask run.

So, let’s first add gunicorn to requirements.txt:

flask
redis
gunicorn

Then remove the last 3 instructions of Dockerfile and add a new CMD instruction:

FROM python:3.7-alpine

WORKDIR /code

RUN apk add --no-cache gcc musl-dev linux-headers

COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt

EXPOSE 5000

COPY . .

CMD ["gunicorn", "--bind", "0.0.0.0:5000", "app:app"]

Images built from this Dockerfile will now run gunicorn instead of the flask dev server.

Once we’ve made the changes, we need to rebuild the web image:

docker compose build

Then we restart the web service to use the new image:

docker compose up web -d --no-deps -t 1

NOTE:

• --no-deps tells Docker not to (re)start dependent services.
• -t 1 specifies the container shutdown timeout of 1 second instead of the default 10s.

Try to reach web again:

curl localhost:8000

It should work as expected.

But if you check the logs:

docker compose logs web -f

You will notice we don’t have the warning anymore since we’re using gunicorn.

Noice!

Step 5: Clean up

To remove all the services, we simply run:

docker compose down

If we had defined volumes in the services (like in the WordPress example), the volumes wouldn’t be automatically removed with docker compose down. This is mainly to avoid accidental deletion of data.

To tear down everything including volumes, use the -v flag:

docker compose down -v

Congratulations! You can now Compose! 🙌

When to use Docker Compose? - use cases

• *Development environments*

  When developing software, the ability to run applications and their dependencies in isolated environments is crucial. For example, you might have a dependency on another team’s application, which in turn might have its own set of complexities like configuring the database in a particular way. By using Compose, you can run the whole stack or remove it with a single command.

• *Automated testing environments*

  Automated workflows like CI/CD pipelines generally require tools to easily create and destroy environments. Containers are ideal for such environments due to their low resource footprint and speed. By using a configuration file, Compose provides a convenient way to create and destroy such environments for your test suite.

• *Single host deployments*

  Although Compose was mainly developed for development and testing workflows, it is sometimes used in production for running containers on a single host. While Compose is improving, it is not an orchestrator like Swarm or Kubernetes, but more of a wrapper around Docker’s API. Check out the official documentation before using compose in production.

Conclusion

In this tutorial, we learnt what Docker Compose is, why it exists, how to use it and when to use it.

By specifying container configuration in a file, Compose simplifies running multi-container workloads on a single host.

If you found this tutorial helpful, leaving a like, comment or subscribing will really help us out.

You might also enjoy our YouTube Channel.

Thanks for making it so far!

We have a LOT more exciting DevOps content on the way! 🙌

See you at the next one.

Till then…

Be bold and keep learning.

But most importantly,

Tech care!