This post describes steps to set up expendable full stack denvironment. What's a denvironment, you may ask? It's development environment. That is just tooooo long to say and write:)

Take time and prepare your dev machine if you want to play along right away.

Description of the project

This project with made-up name "World's largest bass players database" consists of:

• ReactJS frontend
• SailsJS JSON API
• MongoDB for database
• RabbitMQ for queue and async processing
• Redis for cache
• Nginx for reverse proxy that fronts the API.

Let's call it "players", for short.

Let this project have it's main git repository be at https://github.com/svenvarkel/players

(it's time to create yours, now).

Pre-requisites

1. Create 2 names in your /etc/hosts file.

   # /etc/hosts
   
   127.0.0.1 api.players.local #for the API
   127.0.0.1 app.players.local #for the web APP
   

2. Install Docker Desktop

Get it from here and follow the instructions.

Directory layout

The directory layout reflects the stack. On top level there are all familiar names that help the developer to navigate to a component quickly and not waste time on searching for things in obscurely named subfolders or elsewhere. Also - each component is a real component, self-containing and complete. All output or config files or anything that a component would need are placed into the component's directory.

The folder of your development projects is the /.

So here is the layout:

/
/api
    /sails bits and pieces
    /.dockerignore
    /Dockerfile
/mongodb
/nginx
    /Dockerfile
    /conf.d/
        /api.conf
        /app.conf
/rabbitmq
/redis
/web
    /react bits and pieces
    /.dockerignore
    /Dockerfile
/docker-compose.yml

It is all set up as an umbrella git repository with api and web as git submodules. Nginx, MongoDB, Redis and RabbitMQ don't need to have their own repositories.

From now on you have choice either to clone my demo repository or create your own.

If you decide to use my example repository, then run commands:

git clone git@github.com:svenvarkel/players.git
cd players
git submodule init
git submodule update

Steps

First step - create docker-compose.yml

In docker-compose.yml you define your stack in full.

version: "3.7"
services:
  rabbitmq:
    image: rabbitmq:3-management
    environment:
      RABBITMQ_DEFAULT_VHOST: "/players"
      RABBITMQ_DEFAULT_USER: "dev"
      RABBITMQ_DEFAULT_PASS: "dev"
    volumes:
      - type: volume
        source: rabbitmq
        target: /var/lib/rabbitmq/mnesia
    ports:
      - "5672:5672"
      - "15672:15672"
    networks:
      - local
  redis:
    image: redis:5.0.5
    volumes:
      - type: volume
        source: redis
        target: /data
    ports:
      - "6379:6379"
    command: redis-server --appendonly yes
    networks:
      - local
  mongodb:
    image: mongo:4.2
    ports:
      - "27017:27017"
    environment:
      MONGO_INITDB_DATABASE: "admin"
      MONGO_INITDB_ROOT_USERNAME: "root"
      MONGO_INITDB_ROOT_PASSWORD: "root"
    volumes:
      - type: bind
        source: ./mongodb/docker-entrypoint-initdb.d
        target: /docker-entrypoint-initdb.d
      - type: volume
        source: mongodb
        target: /data
    networks:
      - local
  api:
    build: ./api
    image: players-api:latest
    ports:
      - 1337:1337
      - 9337:9337
    environment:
      PORT: 1337
      DEBUG_PORT: 9337
      WAIT_HOSTS: rabbitmq:5672,mongodb:27017,redis:6379
      NODE_ENV: development
      MONGODB_URL: mongodb://dev:dev@mongodb:27017/players?authSource=admin
    volumes:
      - type: bind
        source: ./api/api
        target: /var/app/current/api
      - type: bind
        source: ./api/config
        target: /var/app/current/config
    networks:
      - local
    depends_on:
      - "rabbitmq"
      - "mongodb"
      - "redis"
  web:
    build: ./web
    image: players-web:latest
    ports:
      - 3000:3000
    environment:
      REACT_APP_API_URL: http://api.players.local
    volumes:
      - type: bind
        source: ./web/src
        target: /var/app/current/src
      - type: bind
        source: ./web/public
        target: /var/app/current/public
    networks:
      - local
    depends_on:
      - "api"
  nginx:
    build: nginx
    image: nginx-wait:latest
    restart: on-failure
    environment:
      WAIT_HOSTS: api:1337,web:3000
    volumes:
      - type: bind
        source: ./nginx/conf.d
        target: /etc/nginx/conf.d
      - type: bind
        source: ./nginx/log
        target: /var/log/nginx
    ports:
      - 80:80
    networks:
      - local
    depends_on:
      - "api"
      - "web"
networks:
  local:
    driver: overlay

volumes:
  rabbitmq:
  redis:
  mongodb:

A few comments about features and tricks used here.

My favorite docker trick that I learnt just a few days ago is the use of wait. You will see it in api and nginx Dockerfiles. It's a special app that let's the docker container wait for dependencies until a service actually comes available at a port. The Docker's own "depends_on" is good but it just waits until a dependence container becomes available, not when the actual service is started inside a container. For example - rabbitmq is quite slow to start and it may cause the API behave erratically if it starts up before rabbitmq or mongodb have been fully started.

The second trick you'll see in docker-compose.yml is the use of bind mounts. The code from the dev machine is mounted as a folder inside docker container. It's good for rapid development. Whenever the source code is changed in the editor on developer machine the SailsJS application (or actually - nodemon) in container can detect the changes and restart the application. More details about setting up SailsJS app will follow in future posts, I hope.

Second step - create API and add it as git submodule

sails new api --fast
cd api
git init
git remote add origin <your api repo origin>
git add .
git push -u origin master

Then create Dockerfile for API project:

FROM node:10

ADD https://github.com/ufoscout/docker-compose-wait/releases/download/2.6.0/wait /wait
RUN chmod +x /wait
RUN mkdir -p /var/app/current

# Copy application sources
COPY . /var/app/current

WORKDIR /var/app/current

RUN npm i

RUN chown -R node:node /var/app/current
USER node

# Set the workdir /var/app/current

EXPOSE 1337

# Start the application
CMD /wait && npm run start

Then move up and add it as your main project's submodule

cd ..
git submodule add <your api repo origin> api

Third step - create web app and add it as git submodule

This step is almost a copy of step 2, but it's necessary.

npx create-react-app my-app
cd web
git init
git remote add origin <your web repo origin>
git add .
git push -u origin master

Then create Dockerfile for WEB project:

FROM node:10

ADD https://github.com/ufoscout/docker-compose-wait/releases/download/2.6.0/wait /wait
RUN chmod +x /wait
RUN mkdir -p /var/app/current

# Copy application sources
COPY . /var/app/current

WORKDIR /var/app/current

RUN npm i

RUN chown -R node:node /var/app/current
USER node

# Set the workdir /var/app/current

EXPOSE 3000

# Start the application
CMD /wait && npm run start

As you can see the Dockerfiles for api and web are almost identical. Only the port number is different.

Then move up and add it as your main project's submodule

cd ..
git submodule add <your web repo origin> web

For both projects, api and web, it's also advisable to create .dockerignore file with just two lines:

node_modules
package-lock.json

We want the npm modules inside the container being built fresh every time we rebuild the docker container.

It's time for our first smoke test!

Run docker-compose:

docker-compose up

After Docker grinding a while you should have a working stack! It doesn't do much yet but it's there.

Check with docker-compose:

$ docker-compose ps
   Name                     Command               State                                                                   Ports
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
players_api_1        docker-entrypoint.sh /bin/ ...   Up      0.0.0.0:1337->1337/tcp, 0.0.0.0:9337->9337/tcp
players_mongodb_1    docker-entrypoint.sh mongod      Up      0.0.0.0:27017->27017/tcp
players_nginx_1      /bin/sh -c /wait && exec n ...   Up      0.0.0.0:80->80/tcp
players_rabbitmq_1   docker-entrypoint.sh rabbi ...   Up      0.0.0.0:15671->15671/tcp, 0.0.0.0:15672->15672/tcp, 0.0.0.0:25672->25672/tcp, 4369/tcp, 0.0.0.0:5671->5671/tcp, 0.0.0.0:5672->5672/tcp
players_redis_1      docker-entrypoint.sh redis ...   Up      0.0.0.0:6379->6379/tcp
players_web_1        docker-entrypoint.sh /bin/ ...   Up      0.0.0.0:3000->3000/tcp

As you can see you have:

• API running on port 1337 (9337 also exposed for debugging)
• MongoDB running on port 27017
• RabbitMQ running on many ports, where AMQP port 5672 is of our interest. 15672 is for management - check it out in your browser (use dev as username and password)!
• Redis running on port 6379
• Web app running on port 3000
• Nginx running on port 80.

Nginx proxies both API and web app. So now it's time to give it a look in your browser.

Open http://api.players.local

There it is!

Open http://app.players.local

And there is the ReactJS app.

With this post we won't go into depths of the applications but we focus rather on stack and integration.

So how can services access each other in this Docker setup, you may ask.

Right - it's very straightforward - the services can access each other on a common shared network by calling each other with exactly the same names that are defined in docker-compose.yml.

Redis is at "redis:6379", MongoDB is at "mongodb:27017" etc.

See docker-compose.yml for a tip on how to connect your SailsJS API to MongoDB.

A note about storage

You may have a question like "where is mongodb data stored?". There are 3 volumes defined in docker-compose.yml:

mongodb
redis
rabbitmq

These are special docker volumes that hold the data for each component. It's convenient way of storing data outside of application container but still under control and management of Docker.

A word of warning

There's something I learnt the hard way (not that hard, though) during my endeavour towards full stack dev env. I used command

docker-compose up

lightly and it created temptation to use command

docker-compose down

as lightly because "what goes up must come down", right? Not so fast! Beware that if you run docker-compose down it will destroy your stack including data volumes. So - be careful and better read docker-compose manuals first. Use docker-compose start, stop and restart.

Wrapping it up

More details could follow in similar posts in the future if there's interest for such guides. Shall I continue to add more examples on how to integrate RabbitMQ and Redis within such stack, perhaps? Let me know.

Conclusion

In this post there is a step by step guide on how to set up full stack SailsJS/ReactJS application denvironment (development environment) by using Docker. The denvironment consists of multiple components that are integrated with the API - database, cache and queue. User-facing applications are fronted by the Nginx reverse proxy.