Creating a Microservices App with Dockerized Express Gateway.

In the time of creating software products that need to be highly scalable, microservices app looks like the most simple solution to them all. During my journey as a software developer and working in a project which requires the software to accommodate nearly a million users on demand, I have seen and felt that microservices solve the ultimate problem of proper utilisation of resources and increase the scalability of different services. This removes coupling between different services and help them run independently.

What is an API Gateway?

One of the most fundamental requirement of creating a microservices is the API gateway. An API gateway is an infrastructure layer that sit in front of the microservices. Its purpose is to serve requests from the client by routing it to the right microservice.

This article is about how to create a very simple Node.js microservices application with the help of Dockerized Express API Gateway

Pre-Requisite

This sample project requires few things.

Docker (V19.03.2 or above) installed in your machine.
Node v10 or above.
Basic knowledge of Docker commands.
Ngrok or any other free tunnelling services for exposing localhost to the web

I am doing this project in Mac OSX 11.4.

Getting the Docker Image of Express Gateway and running it.

Run this below command in your terminal to pull the docker image.

docker pull express-gateway

After pulling the image, create a folder and name it Config. Open terminal in that folder and run the below commands.

touch gateway.config.yml
touch system.config.yml

Firstly copy and paste the below YAML code in gateway.config.yml. (I used Sublime Text editor to do this, but any other text editor is fine)

http:
  port: 8080
admin:
  port: 9876
  host: localhost
apiEndpoints:
  api:
    host: localhost
    paths: '/ip'
serviceEndpoints:
  httpbin:
   url: 'https://httpbin.org'
policies:
  - basic-auth
  - cors
  - expression
  - key-auth
  - log
  - oauth2
  - proxy
  - rate-limit
pipelines:
  default:
    apiEndpoints:
      - api
    policies:
    # Uncomment `key-auth:` when instructed to in the Getting Started guide.
    # - key-auth:
      - proxy:
          - action:
              serviceEndpoint: httpbin 
              changeOrigin: true

Copy and paste the YAML code below in system.config.yml file.

# Core
db:
  redis:
    emulate: true
    namespace: EG

#plugins:
  # express-gateway-plugin-example:
  #   param1: 'param from system.config' 

crypto:
  cipherKey: sensitiveKey
  algorithm: aes256
  saltRounds: 10

# OAuth2 Settings
session:
  secret: keyboard cat
  resave: false
  saveUninitialized: false
accessTokens:
  timeToExpiry: 7200000
refreshTokens:
  timeToExpiry: 7200000
authorizationCodes:
  timeToExpiry: 300000

Now create another folder, named models, in Config. Open terminal at models and run the below commands to create three files.

touch applications.json
touch credentials.json
touch users.json

Copy and paste the below code in applications.json.

{
  "$id": "http://express-gateway.io/models/applications.json",
  "type": "object",
  "properties": {
    "name": {
      "type": "string"
    },
    "redirectUri": {
      "type": "string",
      "format": "uri"
    }
  },
  "required": [
    "name"
  ]
}

After that copy and paste the below code in credentials.json

{
  "$id": "http://express-gateway.io/models/credentials.json",
  "type": "object",
  "definitions": {
    "credentialBase": {
      "type": "object",
      "properties": {
        "autoGeneratePassword": {
          "type": "boolean",
          "default": true
        },
        "scopes": {
          "type": [
            "string",
            "array"
          ],
          "items": {
            "type": "string"
          }
        }
      },
      "required": [
        "autoGeneratePassword"
      ]
    }
  },
  "properties": {
    "basic-auth": {
      "allOf": [
        {
          "$ref": "#/definitions/credentialBase"
        },
        {
          "type": "object",
          "properties": {
            "passwordKey": {
              "type": "string",
              "default": "password"
            }
          },
          "required": [
            "passwordKey"
          ]
        }
      ]
    },
    "key-auth": {
      "type": "object",
      "properties": {
        "scopes": {
          "type": [
            "string",
            "array"
          ],
          "items": {
            "type": "string"
          }
        }
      }
    },
    "jwt": {
      "type": "object",
      "properties": {}
    },
    "oauth2": {
      "allOf": [
        {
          "$ref": "#/definitions/credentialBase"
        },
        {
          "type": "object",
          "properties": {
            "passwordKey": {
              "type": "string",
              "default": "secret"
            }
          },
          "required": [
            "passwordKey"
          ]
        }
      ]
    }
  }
}

Lastly copy and paste the below code for users.json

{
  "$id": "http://express-gateway.io/models/users.json",
  "type": "object",
  "properties": {
    "firstname": {
      "type": "string"
    },
    "lastname": {
      "type": "string"
    },
    "username": {
      "type": "string"
    },
    "email": {
      "type": "string",
      "format": "email"
    },
    "redirectUri": {
      "type": "string",
      "format": "uri"
    }
  },
  "required": [
    "username",
    "firstname",
    "lastname"
  ]
}

For this article, we will only focus on configuring the gateway.config.yml file. I will walk through that after setting up and running the docker.

Now that our Config folder is all ready, run the below Docker command to spin up a Docker container.

docker run -d --name express-gateway \ -v /Users/naseef/Config:/var/lib/eg \ -p 8080:8080 \ -p 9876:9876 \ express-gateway

In order for our Express Gateway Docker container to work properly we need to mount a volume with configuration files and volumes. The Config folder will be mounted and it contains all the configuration files and volumes inside it. Please replace '/Users/naseef/Config' in the above Docker to your own path of Config folder.

This should start and run the Docker container named express-gateway. To make sure it is running, run docker ps in the terminal and check.

Creating two simple different microservices with two endpoints

We are going to create two GET Endpoints to act as our microservices.

Create a folder in any directory of your choice named Actor. Setup an Express server in this folder. I used npm init command to setup my package.json, but you can choose other ways to setup the Express server.

Upon creating the package.json, run npm install express --save in Actor to install Express.

Create a file named actor.js and paste the code below.

let express = require('express');
let app = express();
app.listen(3000, () => {
 console.log("Server running on port 3000");
});

app.get("/actors", (req, res, next) => {
    let array_actors = [
        "Tom Cruise",
        "Johnny Depp",
        "Di Caprio",
        "Russel Crowe",
        "Tom Hanks"
    ];
    res.json(array_actors);
});

Run node actor.js to start this server on port 3000 of localhost. Hit the browser (or use Postman) with this url.

http://localhost:3000/actors.

The result should be like this.

[
    "Tom Cruise",
    "Johnny Depp",
    "Di Caprio",
    "Russel Crowe",
    "Tom Hanks"
]

In the same way as above create Movie folder, setup an Express server and create a file named movie.js. Add the below code in the movie.js file

let express = require('express');
let app = express();
app.listen(8000, () => {
 console.log("Server running on port 8000");
});
app.get("/movies", (req, res, next) => {
    let array_movies = [
        "Mission Impossible",
        "Pirates of Carribean",
        "Inception",
        "Gladiator",
        "The Terminal"
    ];
    res.json(array_movies);
});

Run node movie.js to start this server on port 8000 of localhost. Hit the browser (or use Postman) with this url.

http://localhost:8000/movies.

The output should be something like this.

[
    "Mission Impossible",
    "Pirates of Carribean",
    "Inception",
    "Gladiator",
    "The Terminal"
]

Ngrok to expose the GET Endpoints in the web.

Use Ngrok to expose port 3000 and 8000 in the web. If you have trouble running two Ngrok sessions at the same time, follow this link.

I will leave the details of exposing the API in the web through Ngrok out of the scope of this tutorial.

Upon exposing the endpoints with Ngrok, we have the below endpoints for the services.

Movie services: http://412143bfb37a.ngrok.io/movies
Actor services: http://eed882f0ffe3.ngrok.io/actors

Now that we have two running microservices, it is time to configure our gateway.config.yml file to route both these services through our Dockerized Gateway.

Configure gateway.config.yml file of Express Gateway

The Express Gateway accepts requests from the clients and directs them to the microservice in charge of the particular request. For example, if a request is made through the gateway, http://localhost:8080/actors, the request is directed to the actor microservice and a request such as http://localhost:8080/movies is directed to the movie microservice. In summary, each request is accepted through a server and directed to their various host. Let’s set up our Docker to implement this.

Expose the endpoints to the gateway

Navigate into Config folder
Open gateway.config.yml file.

In the apiEndpoints section, we create an API endpoint named “actors”, define the host and path. The path defined is an array of paths we would like to match. This is to cover for all URLs that match the path pattern. Repeat the same for the movies endpoint. For further details about matching patterns for hostname and paths, check out the Express Gateway endpoint documentation.

With these, the Express gateway can accept external APIs(request from the client) of these formats ‘http://localhost:8080/actors’ or ‘http://localhost:8080/actors/*’

Create the Service Endpoints to the gateway

The service endpoints are the endpoints of our microservices, in this case, the actor and the movie microservices. The external request from the API endpoints is directed to the service endpoints. In the serviceEndpoints section still in the same gateway.config.yml file, we create services and define their URLs. For actors, we call its service endpoints actorService and add its endpoints ‘http://eed882f0ffe3.ngrok.io’. Repeat the same for the movie service endpoint.

Finishing up this long post!

Now let’s tie up the API endpoints and the service endpoints. It is configured in the Pipelines section. This is where we connect the API endpoints and the service endpoints.

You will find that a default has been pre-defined, so we need to just define ours. You can copy and paste the default pipeline and then change some of the details. We name our pipeline “actorPipeline”, add the endpoint name which is “actors”. Find the serviceEndpoint in the proxy policy of the actorPipeline pipeline and add our service endpoint name which is “actorService”. Repeat the procedure for the movie pipeline.

http:
  port: 8080
admin:
  port: 9876
  host: localhost
apiEndpoints:
  api:
    host: localhost
    paths: '/ip'
  actors:
    host: localhost
    paths: ['/actors','/actors/*']
  movies:
    host: localhost
    paths: ['/movies','/movies/*'] 

serviceEndpoints:
  httpbin:
   url: 'https://httpbin.org'
  actorService:
   url: 'http://eed882f0ffe3.ngrok.io'
  movieService:
   url: 'http://412143bfb37a.ngrok.io'   
policies:
  - basic-auth
  - cors
  - expression
  - key-auth
  - log
  - oauth2
  - proxy
  - rate-limit
pipelines:
  actorPipeline:
    apiEndpoints:
      - actors
    policies:
    # Uncomment `key-auth:` when instructed to in the Getting Started guide.
    # - key-auth:
      - proxy:
          - action:
              serviceEndpoint: actorService 
              changeOrigin: true
  moviePipeline:
    apiEndpoints:
      - movies
    policies:
    # Uncomment `key-auth:` when instructed to in the Getting Started guide.
    # - key-auth:
      - proxy:
          - action:
              serviceEndpoint: movieService 
              changeOrigin: true
  default:
    apiEndpoints:
      - api
    policies:
    # Uncomment `key-auth:` when instructed to in the Getting Started guide.
    # - key-auth:
      - proxy:
          - action:
              serviceEndpoint: httpbin 
              changeOrigin: true

Now our whole system is ready. Save the gateway.config.yml file and restart the Docker container.

docker restart express-gateway

The Ngrok and Node services should be running. If all are functioning properly we should be getting proper output by using our URLS

http://localhost:8080/actors
http://localhost:8080/movies

This is the end of this Microservice App creation with Dockerized Express-gateway. If you face any errors or any difficulties feel free to comment below.

Thank you. Happy Coding

Blog