How to setup a MongoDB Replica set for development using Docker

In this post I will be sharing how I configure and run a MongoDB replica set on my local machine, using Docker.

While there are ample examples on the internet about running MongoDB containers, not many of them focus on how to bring up a MongoDB replica set.

Doing development against single node MongoDB is fine for starting, but as one starts using features like transactions, it becomes imperative that the MongoDB deployment has replica set enabled (or sharded cluster, a topic for future discussion).

Creating and developing against a MongoDB replica set in local development setup helps you learn the topic, and be more confident that your application code is already adapted to run against replica sets in production.

To follow along, you will need to have Docker, and Docker Compose installed in your machine.

To install Docker, I suggest following the official documentation at this link.

Once Docker is installed, install Docker Compose by following the links from this link.

I have created this Github repository, which contains all the required artifacts to follow along with this tutorial.

The Docker Compose file

Once Docker and Docker Compose are both installed on your machine, we can start writing the Docker Compose file, that contains directives to bring up the MongoDB replica set. I will show the full docker-compose.yml file first, and then explain the individual details.

version: '3.9'  # Docker Engine release 19.03.0+ [https://docs.docker.com/compose/compose-file/]

services: 

    # setup MongoDB cluster for production
    mongo-replica-setup:
        container_name: mongo-setup
        image: 'mongo:4.2'
        restart: on-failure
        networks:
            - netApplication
        volumes:
        - ./.docker/mongodb/scripts/mongosetup.sh:/scripts/mongosetup.sh
        entrypoint: ["bash", "/scripts/mongosetup.sh" ]
        env_file:
            - .env
        environment:
            MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
            MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
        depends_on:
        - mongo1
        - mongo2
        - mongo3

    mongo1:
        hostname: 'mongo1'
        container_name: 'mongo1'
        image: 'mongo:4.2'
        restart: 'on-failure'
        command: ["-f", "/etc/mongod.conf", "--keyFile", "/auth/file.key", "--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]
        expose: 
            - 27017
        ports: 
            - 30001:27017 
        networks: 
            - netApplication
        volumes:
            - dataMongo1:/data/db
            - logMongo1:/var/log/mongodb
            - ./.docker/mongodb/initdb.d/:/docker-entrypoint-initdb.d/
            - ./.docker/mongodb/mongod.conf:/etc/mongod.conf
            - ./.docker/mongodb/file.key:/auth/file.key
        healthcheck:
            test: test $$(echo "rs.status().ok" | mongo -u $${MONGO_INITDB_ROOT_USERNAME} -p $${MONGO_INITDB_ROOT_PASSWORD} --quiet) -eq 1
            interval: 30s
            start_period: 60s
        env_file:
            - .env
        environment:
            MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
            MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
            MONGO_INITDB_DATABASE: ${MONGO_INITDB_DATABASE}

    mongo2:
        hostname: 'mongo2'
        container_name: 'mongo2'
        image: 'mongo:4.2'
        command: ["-f", "/etc/mongod.conf", "--keyFile", "/auth/file.key", "--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]
        restart: 'on-failure'
        expose: 
            - 27017
        ports: 
            - 30002:27017  
        networks: 
            - netApplication
        volumes:
            - dataMongo2:/data/db
            - logMongo2:/var/log/mongodb
            - ./.docker/mongodb/mongod.conf:/etc/mongod.conf
            - ./.docker/mongodb/file.key:/auth/file.key
        env_file:
            - .env
        environment:
            MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
            MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
            MONGO_INITDB_DATABASE: ${MONGO_INITDB_DATABASE}
        depends_on: 
            - mongo1

    mongo3:
        hostname: 'mongo3'
        container_name: 'mongo3'
        image: 'mongo:4.2'
        command: ["-f", "/etc/mongod.conf", "--keyFile", "/auth/file.key", "--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]
        restart: 'on-failure'
        expose: 
            - 27017
        ports: 
            - 30003:27017  
        networks: 
            - netApplication
        volumes:
            - dataMongo3:/data/db
            - logMongo3:/var/log/mongodb
            - ./.docker/mongodb/mongod.conf:/etc/mongod.conf
            - ./.docker/mongodb/file.key:/auth/file.key
        env_file:
            - .env
        environment:
            MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
            MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
            MONGO_INITDB_DATABASE: ${MONGO_INITDB_DATABASE}
        depends_on: 
            - mongo1

volumes: 
    dataMongo1:
    dataMongo2:
    dataMongo3:
    logMongo1:
    logMongo2:
    logMongo3:

networks: 
    netApplication:

The Setup

Here we are trying to setup a MongoDB replica set with 3 nodes.

One of these nodes will be the Primary node - all writes to the database will happen via this node.

The remaining two, are Secondary nodes, providing two-levels of data replication in the setup.

A solid explanation of how data replication works in MongoDB can be found at this link.

Exploring the docker-compose file

Coming back to the yaml file, let's go through the service declaration of mongo1 and see what we are trying to achieve.

The basic stuff

mongo1:
    hostname: 'mongo1'
    container_name: 'mongo1'
    image: 'mongo:4.2'
    restart: 'on-failure'
    :
    expose: 
        - 27017
    ports: 
        - 30001:27017 
    networks: 
        - netApplication

Nothing fancy here, we are targetting MongoDB server version 4.2 to run these containers.

We have declared hostname and container name, and set the container restart policy to on-failure.

On the ports front, we have exposed 27017 to other docker containers running on the same docker network. Also, this port is mapped to host port 30001.

Lastly, we will be running our replica set on a named network.

Starting the mongod instance

command: ["-f", "/etc/mongod.conf", "--keyFile", "/auth/file.key", "--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]

This will start the mongod service with the following:

-f /etc/mongod.conf
- specifies that the runtime configuration options should be picked up from the provided mongod.conf file.
- we will see the contents of this file later
-keyFile /auth/file.key
- In our replica set, the nodes will use the contents of a shared keyfile to authenticate to each other.
- Here we are specifying the path to the keyfile
--replSet ${MONGO_REPLICA_SET_NAME}
- This configures the node to run in a replica set.
- We are also specifying the name of the replica set, which is to be picked up from the environment variable ${MONGO_REPLICA_SET_NAME}
--bind_ip_all
- This specifies the mongod instance to bind to all IPv4 addresses (0.0.0.0).

Container volumes

Before proceeding it is worth mentioning again, that I have created this Github repository to house all the artifacts mentioned here.

volumes:
    - mongoData1:/data/db
    - mongoLog1:/var/log/mongodb
    - ./.docker/mongodb/initdb.d/:/docker-entrypoint-initdb.d/
    - ./.docker/mongodb/mongod.conf:/etc/mongod.conf
    - ./.docker/mongodb/file.key:/auth/file.key

Here we have setup a few Docker managed volumes to store the container's data in host's file system.

mongoData1: This will act as the peristent store for MongoDB's data.
mongoLog1: All of MongoDB's logs for this container will be stored here.
./.docker/mongodb/initdb.d/
- When mongod container starts for the very first time, it will look for a directory named /docker-entrypoint-initdb.d, and execute all files with extensions .sh and .js.
- We map this directory to ./.docker/mongodb/initdb.d/ in host.
- As we will see later, we will use this directory to store a bash script that allows us to create a user.
./.docker/mongodb/mongod.conf
- We are essentially providing the path (in host) from where MongoDB will pick up its config file.
./.docker/mongodb/file.key
- Like above volume, here we are specifying the keyfile that this MongoDB node will be using to authenticate itself to other nodes in the replica set.
- Steps to create this keyfile is provided in the Github repository that I have created.

env_file:
    - .env
environment:
    MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
    MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
    MONGO_INITDB_DATABASE: ${MONGO_INITDB_DATABASE}

Here we have specified that the docker compose will be using the .env file to read environment variables and inject them into the container.

This allows us to specify that the database's root username/password and initial database name to be picked up from specific environment variables.

In my github repository, I have provided a sample env file that you can use to create your own .env file.

Contents of the sample env file looks like this:

# MongoDB
MONGO_URL=mongodb://mongodb:27017
MONGO_INITDB_ROOT_USERNAME=<root_username>
MONGO_INITDB_ROOT_PASSWORD=<root_password>
MONGO_INITDB_DATABASE=<app_db-name>
MONGO_INITDB_USERNAME=<app_username>
MONGO_INITDB_PASSWORD=<app_password>
MONGO_REPLICA_SET_NAME=rs0

One must make sure not to commit this file in github, as these contains secrests like root username/password.

The above image provides a tree view of the files and directories discussed here.

Health Check

healthcheck:
    test: test $$(echo "rs.status().ok" | mongo -u $${MONGO_INITDB_ROOT_USERNAME} -p $${MONGO_INITDB_ROOT_PASSWORD} --quiet) -eq 1
    interval: 30s
    start_period: 60s

Here we are specifying the healthcheck command that docker will use to check if the container is up and running.

To summarize, the mongo1 service's specification looks like this:

mongo1:
    hostname: 'mongo1'
    container_name: 'mongo1'
    image: 'mongo:4.2'
    restart: 'on-failure'
    command: ["-f", "/etc/mongod.conf", "--keyFile", "/auth/file.key", "--replSet", "${MONGO_REPLICA_SET_NAME}", "--bind_ip_all"]
    expose: 
        - 27017
    ports: 
        - 30001:27017 
    networks: 
        - netApplication
    volumes:
        - mongoData1:/data/db
        - mongoLog1:/var/log/mongodb
        - ./.docker/mongodb/initdb.d/:/docker-entrypoint-initdb.d/
        - ./.docker/mongodb/mongod.conf:/etc/mongod.conf
        - ./.docker/mongodb/file.key:/auth/file.key
    healthcheck:
        test: test $$(echo "rs.status().ok" | mongo -u $${MONGO_INITDB_ROOT_USERNAME} -p $${MONGO_INITDB_ROOT_PASSWORD} --quiet) -eq 1
        interval: 30s
        start_period: 60s
    env_file:
        - .env
    environment:
        MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
        MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
        MONGO_INITDB_DATABASE: ${MONGO_INITDB_DATABASE}

The other two nodes - mongo1 and mongo2 is a subset of this specification. The only things missing are the healthcheck section, and a volume that maps to docker-entrypoint-initdb.d. This is because we have designed this setup such that mongo1 will become the primary and the rest secondary.

Being secondary nodes, they dont need the initdb part to work.

Create the Mongo Cluster

Once all the nodes in the replica set is up, we will need to initiate Replica set configuration on these nodes.

We will be using a 4th container - mongo-replica-setup to do this.

mongo-replica-setup:
    container_name: mongo-setup
    image: 'mongo:4.2'
    restart: on-failure
    networks:
        - netApplication
    volumes:
    - ./.docker/mongodb/scripts/mongosetup.sh:/scripts/mongosetup.sh
    entrypoint: ["bash", "/scripts/mongosetup.sh" ]
    env_file:
        - .env
    environment:
        MONGO_INITDB_ROOT_USERNAME: ${MONGO_INITDB_ROOT_USERNAME}
        MONGO_INITDB_ROOT_PASSWORD: ${MONGO_INITDB_ROOT_PASSWORD}
    depends_on:
        - mongo1
        - mongo2
        - mongo3

Once this container starts, it will connect to mongo1, and execute a script that will initiate a replica set over mongo1, mongo2 and mongo3.

Once the script is executed this container exits.

Its important to note that we have set the depends_on field of this container to mongo1, mongo2 and mongo3, which ensures that this container will start only after all those 3 containers are up. This is important, as you wont want to run the script without all the nodes of the replica set being up.

The following is the script that this container runs, to initiate the replica set:

#!/bin/bash

MONGODB1=mongo1
MONGODB2=mongo2
MONGODB3=mongo3

echo "**********************************************" ${MONGODB1}
echo "Waiting for startup.."
sleep 30
echo "done"

echo SETUP.sh time now: `date +"%T" `
mongo --host ${MONGODB1}:27017 -u ${MONGO_INITDB_ROOT_USERNAME} -p ${MONGO_INITDB_ROOT_PASSWORD} <<EOF
var cfg = {
    "_id": "rs0",
    "protocolVersion": 1,
    "version": 1,
    "members": [
        {
            "_id": 0,
            "host": "${MONGODB1}:27017",
            "priority": 2
        },
        {
            "_id": 1,
            "host": "${MONGODB2}:27017",
            "priority": 0
        },
        {
            "_id": 2,
            "host": "${MONGODB3}:27017",
            "priority": 0,
        }
    ]
};
rs.initiate(cfg, { force: true });
rs.secondaryOk();
db.getMongo().setReadPref('primary');
rs.status();
EOF

If you look closely, we have set the priority for mongo1 higher than the other two nodes, which ensures that initially, mongo1 will be acting as the Primary node.

Final Result

To start the replica set, run docker-compose up -d from command line.

Once everything comes up, you can run docker-compose status, and see something like this:

To connect to the replica set, you can use mongo client like so:

$ mongo "mongodb://localhost:30001,localhost:30002,localhost:30003/<MONGO_INITDB_DATABASE>" -u <MONGO_INITDB_USERNAME>

Blog