TheLazy.Dev
Portfolio GitHub

Blog

javascript |es6 |node |docker

ES6 by example: a module/CLI to wait for Postgres in docker-compose

hugo__df

Hugo Di Francesco

Posted on July 11, 2018

ES6 by example: a module/CLI to wait for Postgres in docker-compose

When using docker-compose, it’s good practice to make anything that relies on Postgres wait for it to be up before launching. This avoids connection issues inside the app.

This post walks through how to deliver this functionality both as a CLI and a module that works both as a CommonJS module (require) and ES modules, without transpilation.

“A fast, production ready, zero-dependency ES module loader for Node 6+!” is esm’s promise. From this sample project, it’s worked.

This was sent out on the Code with Hugo newsletter last Monday.
Subscribe to get the latest posts right in your inbox (before anyone else).

Writing ES modules without a build step 🎨

To begin we install esm: npm install --save esm.
Next we’ll need a file for our module, wait-for-pg.js:

export const DEFAULT_MAX_ATTEMPTS = 10;
export const DEFAULT_DELAY = 1000; // in ms
Enter fullscreen mode Exit fullscreen mode

Trying to run this file with Node will throw:

$ node wait-for-pg.js
/wait-for-pg/wait-for-pg.js:1
export const DEFAULT_MAX_ATTEMPTS = 10;
^^^^^^

SyntaxError: Unexpected token export
Enter fullscreen mode Exit fullscreen mode

export and import don’t work in Node yet (without flags), the following runs though:

$ node -r esm wait-for-pg.js
Enter fullscreen mode Exit fullscreen mode

That’s if we want to run it as a script, say we want to let someone else consume it via require we’ll need an index.js with the following content:

require = require('esm')(module);
module.exports = require('./wait-for-pg');
Enter fullscreen mode Exit fullscreen mode

We can now run index.js as a script:

$ node index.js
Enter fullscreen mode Exit fullscreen mode

We can also require it:

$ node # start the Node REPL
> require('./index.js')
{ DEFAULT_MAX_ATTEMPTS: 10,
  DEFAULT_DELAY: 1000 }
Enter fullscreen mode Exit fullscreen mode

To tell users wanting to require the package with Node, we can use the "main" field in package.json:

{
  "main": "index.js",
  "dependencies": {
    "esm": "^3.0.62"
  }
}
Enter fullscreen mode Exit fullscreen mode

Sane defaults 🗃

To default databaseUrl, maxAttempts and delay, we use ES6 default parameters + parameter destructuring.
Let’s have a look through some gotchas of default parameters that we’ll want to avoid:

  1. Attempting to destructure ‘null’ or ‘undefined’
  2. ‘null’ remains, undefined gets defaulted

Attempting to destructure null or undefined 0️⃣ 

export const DEFAULT_MAX_ATTEMPTS = 5;
export const DEFAULT_DELAY = 1000; // in ms

export function waitForPostgres({
  databaseUrl = (
    process.env.DATABASE_URL || 
    'postgres://postgres@localhost'
  ),
  maxAttempts = DEFAULT_MAX_ATTEMPTS,
  delay = DEFAULT_DELAY
}) {
  console.log(
    databaseUrl, 
    maxAttempts,
    delay
  )
}
Enter fullscreen mode Exit fullscreen mode

Callings the following will throw:

$ node -r esm # node REPL
> import { waitForPostgres } from './wait-for-pg';
> waitForPostgres()
TypeError: Cannot destructure property `databaseUrl` of 'undefined' or 'null'.
    at waitForPostgres (/wait-for-pg/wait-for-pg.js:4:19)
> waitForPostgres(null)
TypeError: Cannot destructure property `databaseUrl` of 'undefined' or 'null'.
    at waitForPostgres (/wait-for-pg/wait-for-pg.js:4:19)
Enter fullscreen mode Exit fullscreen mode

To avoid this, we should add = {} to default the parameter that’s being destructured (wait-for-pg.js):

export const DEFAULT_MAX_ATTEMPTS = 5;
export const DEFAULT_DELAY = 1000; // in ms

export function waitForPostgres({
  databaseUrl = (
    process.env.DATABASE_URL || 
    'postgres://postgres@localhost'
  ),
  maxAttempts = DEFAULT_MAX_ATTEMPTS,
  delay = DEFAULT_DELAY
} = {}) {
  console.log(
    databaseUrl, 
    maxAttempts,
    delay
  )
}
Enter fullscreen mode Exit fullscreen mode

It now runs:

$ node -r esm # node REPL
> import { waitForPostgres } from './wait-for-pg';
> waitForPostgres()
postgres://postgres@localhost 10 1000
Enter fullscreen mode Exit fullscreen mode

The values were successfully defaulted when not passed a parameter. However the following still errors:

> waitForPostgres(null)
postgres://postgres@localhost 10 1000
TypeError: Cannot destructure property `databaseUrl` of 'undefined' or 'null'.
    at waitForPostgres (/wait-for-pg/wait-for-pg.js:4:19)
Enter fullscreen mode Exit fullscreen mode

‘null’ remains, undefined gets defaulted 🔎 

$ node -r esm # node REPL
> import { waitForPostgres } from './wait-for-pg';
> waitForPostgres({ databaseUrl: null, maxAttempts: undefined })
null 10 1000
Enter fullscreen mode Exit fullscreen mode

The values explicitly set as null doesn’t get defaulted whereas an explicit undefined and an implicit one do, that’s just how default parameters work, which isn’t exactly like the old-school way of writing this:

export const DEFAULT_MAX_ATTEMPTS = 5;
export const DEFAULT_DELAY = 1000; // in ms

export function waitForPostgres(options) {
  const databaseUrl = (
    options && options.databaseUrl ||
    process.env.DATABASE_URL ||
    'postgres://postgres@localhost'
  );
  const maxAttempts = options && options.maxAttempts || DEFAULT_MAX_ATTEMPTS;
  const delay = options && options.delay || DEFAULT_DELAY;
  console.log(
    databaseUrl, 
    maxAttempts,
    delay
  )
}
Enter fullscreen mode Exit fullscreen mode

Which would yield the following:

$ node -r esm # node REPL
> import { waitForPostgres } from './wait-for-pg';
> waitForPostgres({ databaseUrl: null, maxAttempts: undefined })
'postgres://postgres@localhost' 10 1000
Enter fullscreen mode Exit fullscreen mode

Since null is just as falsy as undefined 🙂 .

Waiting for Postgres with async/await 🛎

Time to implement wait-for-pg.
To wait for Postgres we’ll want to:

  • try to connect to it
  • if that fails
    • try again later
  • if that succeeds
    • finish

Let’s install a Postgres client, pg using: npm install --save pg

pg has a Client object that we can pass a database URL to when instantiating it (new Client(databaseUrl)). That client instance has a .connect method that returns a Promise which resolves on connection success and rejects otherwise.
That means if we mark the waitForPostgres function as async, we can await the .connect call.

When await-ing a Promise, a rejection will throw an error so we wrap all the logic in a try/catch.

  • If the client connection succeeds we flip the loop condition so that the function terminates
  • If the client connection fails
    • we increment the retries counter, if it’s above the maximum number of retries (maxAttempts), we throw which, since we’re in an async function throw is the equivalent of doing Promise.reject
    • otherwise we call another function that returns a Promise (timeout) which allows us to wait before doing another iteration of the loop body
  • We make sure to export function waitForPostgres() {}

wait-for-pg.js:

import { Client } from 'pg';

export const DEFAULT_MAX_ATTEMPTS = 10;
export const DEFAULT_DELAY = 1000; // in ms

const timeout = ms => new Promise(
  resolve => setTimeout(resolve, ms)
);

export async function waitForPostgres({
  databaseUrl = (
    process.env.DATABASE_URL || 
    'postgres://postgres@localhost'
  ),
  maxAttempts = DEFAULT_MAX_ATTEMPTS,
  delay = DEFAULT_DELAY
} = {}) {
  let didConnect = false;
  let retries = 0;
  while (!didConnect) {
    try {
      const client = new Client(databaseUrl);
      await client.connect();
      console.log('Postgres is up');
      client.end();
      didConnect = true;
    } catch (error) {
      retries++;
      if (retries > maxAttempts) {
        throw error;
      }
      console.log('Postgres is unavailable - sleeping');
      await timeout(delay);
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Integrating as a CLI with meow 😼

meow is a CLI app helper from Sindre Sohrus, install it: npm install --save meow
Create wait-for-pg-cli.module.js:

import {
  waitForPostgres,
  DEFAULT_MAX_ATTEMPTS,
  DEFAULT_DELAY
} from './wait-for-pg';
import meow from 'meow';

const cli = meow(`
    Usage
      $ wait-for-pg <DATABASE_URL>
    Options
      --max-attempts, -c Maximum number of attempts, default: ${DEFAULT_MAX_ATTEMPTS}
      --delay, -d Delay between connection attempts in ms, default: ${DEFAULT_DELAY}
    Examples
      $ wait-for-pg postgres://postgres@localhost:5432 -c 5 -d 3000 && npm start
      # waits for postgres, 5 attempts at a 3s interval, if
      # postgres becomes available, run 'npm start'
`, {
    inferType: true,
    flags: {
      maxAttempts: {
        type: 'string',
        alias: 'c'
      },
      delay: {
        type: 'string',
        alias: 'd'
      }
    }
  });
console.log(cli.input, cli.flags);
Enter fullscreen mode Exit fullscreen mode

We use inferType so that the values for maxAttempts and delay get converted to numbers instead of being strings.
We can run it using:

$ node -r esm wait-for-pg-cli.module.js
[] {}
Enter fullscreen mode Exit fullscreen mode

The following is a template string, it will replace things inside of ${} with the value in the corresponding expression (in this instance the value of the DEFAULT_MAX_ATTEMPTS and DEFAULT_DELAY variables)

`
  Usage
    $ wait-for-pg <DATABASE_URL>
  Options
    --max-attempts, -c Maximum number of attempts, default: ${DEFAULT_MAX_ATTEMPTS}
    --delay, -d Delay between connection attempts in ms, default: ${DEFAULT_DELAY}
  Examples
    $ wait-for-pg postgres://postgres@localhost:5432 -c 5 -d 3000 && npm start
    # waits for postgres, 5 attempts at a 3s interval, if
    # postgres becomes available, run 'npm start'
`;
Enter fullscreen mode Exit fullscreen mode

To get the flags and first input, wait-for-pg-cli.module.js:

import {
  waitForPostgres,
  DEFAULT_MAX_ATTEMPTS,
  DEFAULT_DELAY
} from './wait-for-pg';
import meow from 'meow';

const cli = meow(`
    Usage
      $ wait-for-pg <DATABASE_URL>
    Options
      --max-attempts, -c Maximum number of attempts, default: ${DEFAULT_MAX_ATTEMPTS}
      --delay, -d Delay between connection attempts in ms, default: ${DEFAULT_DELAY}
    Examples
      $ wait-for-pg postgres://postgres@localhost:5432 -c 5 -d 3000 && npm start
      # waits for postgres, 5 attempts at a 3s interval, if
      # postgres becomes available, run 'npm start'
`, {
    inferType: true,
    flags: {
      maxAttempts: {
        type: 'string',
        alias: 'c'
      },
      delay: {
        type: 'string',
        alias: 'd'
      }
    }
  });
waitForPostgres({
  databaseUrl: cli.input[0],
  maxAttempts: cli.flags.maxAttempts,
  delay: cli.flags.delay,
}).then(
  () => process.exit(0)
).catch(
  () => process.exit(1)
);
Enter fullscreen mode Exit fullscreen mode

If you don’t have a Postgres instance running on localhost the following shouldn’t print Here, that’s thanks to process.exit(1) in the .catch block:

$ node -r esm wait-for-pg-cli.module.js -c 5 && echo "Here"
Postgres is unavailable - sleeping
Postgres is unavailable - sleeping
Postgres is unavailable - sleeping
Postgres is unavailable - sleeping
Postgres is unavailable - sleeping
Enter fullscreen mode Exit fullscreen mode

Packaging and clean up 📤

We can use the "bin" key in package.json to be able to run the command easily:

{
  "main": "index.js",
  "bin": {
    "wait-for-pg": "./wait-for-pg-cli.js"
  },
  "dependencies": {
    "esm": "^3.0.62",
    "meow": "^5.0.0",
    "pg": "^7.4.3"
  }
}
Enter fullscreen mode Exit fullscreen mode

Where wait-for-pg-cli.js is:

#!/usr/bin/env node
require = require("esm")(module/*, options*/);
module.exports = require('./wait-for-pg-cli.module');
Enter fullscreen mode Exit fullscreen mode

Don’t forget to run chmod +x wait-for-pg-cli.js
esm allows us to use top-level await, that means in wait-for-pg-cli.module.js, we can replace:

waitForPostgres({
  databaseUrl: cli.input[0],
  maxAttempts: cli.flags.maxAttempts,
  delay: cli.flags.delay,
}).then(
  () => process.exit(0)
).catch(
  () => process.exit(1)
);
Enter fullscreen mode Exit fullscreen mode

With:

try {
  await waitForPostgres({
    databaseUrl: cli.input[0],
    maxAttempts: cli.flags.maxAttempts,
    delay: cli.flags.delay,
  });
  process.exit(0);
} catch (error) {
  process.exit(1);
}
Enter fullscreen mode Exit fullscreen mode

Running the CLI throws:

$ ./wait-for-pg-cli.js
wait-for-pg/wait-for-pg-cli.module.js:36
  await waitForPostgres({
  ^^^^^

SyntaxError: await is only valid in async function
Enter fullscreen mode Exit fullscreen mode

We need to add "esm" with "await": true in package.json:

{
  "main": "index.js",
  "bin": {
    "wait-for-pg": "./wait-for-pg-cli.js"
  },
  "dependencies": {
    "esm": "^3.0.62",
    "meow": "^5.0.0",
    "pg": "^7.4.3"
  },
  "esm": {
    "await": true
  }
}
Enter fullscreen mode Exit fullscreen mode

This now works:

$ ./wait-for-pg-cli.js -c 1
Postgres is unavailable - sleeping
Enter fullscreen mode Exit fullscreen mode

Extras

This was sent out on the Code with Hugo newsletter last Monday.
Subscribe to get the latest posts right in your inbox (before anyone else).

Publishing to npm with np

  1. Run: npm install --save-dev np
  2. Make sure you have a valid "name" field in package.json, eg. "@hugodf/wait-for-pg"
  3. npx np for npm v5+ or ./node_modules/.bin/np (npm v4 and down)

Pointing to the ESM version of the module

Use the "module" fields in package.json

{
  "name": "wait-for-pg",
  "version": "1.0.0",
  "description": "Wait for postgres",
  "main": "index.js",
  "module": "wait-for-pg.js",
  "bin": {
    "wait-for-pg": "./wait-for-pg-cli.js"
  },
  "dependencies": {
    "esm": "^3.0.62",
    "meow": "^5.0.0",
    "pg": "^7.4.3"
  },
  "devDependencies": {
    "np": "^3.0.4"
  },
  "esm": {
    "await": true
  }
}
Enter fullscreen mode Exit fullscreen mode

A Promise wait-for-pg implementation 

import { Client } from 'pg';

export const DEFAULT_MAX_ATTEMPTS = 10;
export const DEFAULT_DELAY = 1000; // in ms

const timeout = ms => new Promise(
  resolve => setTimeout(resolve, ms)
);

export function waitForPostgres({
  databaseUrl = (
    process.env.DATABASE_URL ||
    'postgres://postgres@localhost'
  ),
  maxAttempts = DEFAULT_MAX_ATTEMPTS,
  delay = DEFAULT_DELAY,
} = {},
  retries = 1
) {
  const client = new Client(databaseUrl);
  return client.connect().then(
    () => {
      console.log('Postgres is up');
      return client.end();
    },
    () => {
      if (retries > maxAttempts) {
        return Promise.reject(error);
      }
      console.log('Postgres is unavailable - sleeping');
      return timeout(delay).then(
        () => waitForPostgres(
          { databaseUrl, maxAttempts, delay },
          retries + 1
        )
      );
    }
  );
}
Enter fullscreen mode Exit fullscreen mode

Matthew Henry

💖 💪 🙅 🚩
hugo__df
Hugo Di Francesco

Posted on July 11, 2018

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

ES6 by example: a module/CLI to wait for Postgres in docker-compose
javascript ES6 by example: a module/CLI to wait for Postgres in docker-compose

July 11, 2018