Koa Js : Part 5 - how to connect a caching database to your Koa Js server

Caching is the storing of data temporarily and retrieving data from a high-performance store (usually memory) either explicitly or implicitly.

Advantage: When using external api's, you may have a restricted number of calls or cost per api call. If your api data doesn't constantly update, this not only reduces the number of api calls made but also reduces the loading speed.

In this tutorial, we'll be storing data temporarily in our Koa JS server using node-cade.

Before you start this part, make sure you have read part 1 of this tutorial series as we're gonna be starting from there.

If you want to follow along, fork or clone part 1 before you continue.

Packages

Let's start by installing the packages that we need:

npm i node-cache axios

What are we installing?

node-cache - this is the package we'll use to store our data temporarily.
axios - this is what we'll use to fetch our data.

Middleware

We'll need to create middleware folder, a cache.js and a crypto.cache.js file, so run the following command in your terminal:

mkdir middleware
touch middleware/cache.js middleware/crypto.cache.js

Now let's add the following to the cache.js file located in the middleware folder:

const NodeCache = require("node-cache");
const shortCache = new NodeCache({ stdTTL: 60 * 15 });

module.exports = {
  shortCache,
};

What is happening?

We've just created a cache and named it shortCache, we've then set the time using stdTTL key. This key is measured with seconds, so in this case we've set 60 seconds multiplied by 15. Therefore, our cache is 15 minutes long. Keep that in mind moving forward.

We'll now add the following to our crypto.cache.js file:

const { shortCache } = require("./cache");

const cryptoListCache = (ctx, next) => {
  try {
    if (shortCache.has("crypto-list")) {
      return (ctx.body = shortCache.get("crypto-list"));
    }
    return next();
  } catch (err) {
    ctx.status = 500;
    console.log(err);
    throw err;
  }
};

module.exports = {
  cryptoListCache,
};

What is happening?

This cache will be used as middleware. If the cache has a key named crypto-list then it will return the value stored in the key. However, if there isn't then the middleware is skipped. This is important to note as later we'll be setting the key and storing a data in the cache.

Helpers

For our api data, we will be using this endpoint to get our crypto data.

We'll only be returning parts of the data so before we make the api call, let's create a helper function to manipulate the response data.

Run the following command:

mkdir helpers
touch helpers/crypto.helpers.js

This should return a directory named helpers which containing a file named crypto.helpers.js.

For each crypto object, we'll only be returning the following keys exchange_id, symbol, price, spread, change_24h, volume_24h and updated_at.

Copy the following code into the crypto.helpers.js in the helpers directory:

const cryptoRender = ({ markets }) => {
  return markets.map((crypto) => {
    const {
      exchange_id,
      symbol,
      price,
      spread,
      change_24h,
      volume_24h,
      updated_at,
    } = crypto;

    return {
      exchangeId: exchange_id,
      symbol,
      price,
      spread,
      "24hChange": change_24h,
      "24hVolume": volume_24h,
      updated_at,
    };
  });
};

module.exports = {
  cryptoRender,
};

This is what we've done:

Deconstruct the keys from each object.
Changed the keys which were underscore to camel case.
Returned only the fields we need from the data

Note: If the when passing a value, if the value has the same name as the key, you don't need to type it twice. For example, we did not need to type symbol: symbol. This applies above to symbol, price, spread and updated_at.

Services

We'll now create some service files to handle external api calls.

Run the following commands:

mkdir services
touch services/request.services.js services/crypto.services.js

We should now have a services directory with two files:

request.services.js - this where we'll use axios to make our api request functions.
crypto.services.js - this where we'll import our request functions to make requests for our crypto data.

So let's add this to your request.services.js file:

const { request } = require("axios");

const getRequest = async (url) => {
  try {
    const res = await request({
      method: "GET",
      url,
    });
    return res.data;
  } catch (err) {
    console.log(err);
    throw err;
  }
};

module.exports = {
  getRequest,
};

What did we do?

We first import request from axios
created a async function named getRequest to handle our api calls
return only the response data from the api call.

Now let's add the following code to your crypto.services.js file:

const { cryptoRender } = require("../helpers/crypto.helpers");
const { getRequest } = require("./request.services");

const getCryptoData = async () => {
  try {
    const resData = await getRequest("https://www.cryptingup.com/api/markets");

    return cryptoRender(resData);
  } catch (err) {
    console.log(err);
    throw err;
  }
};

module.exports = {
  getCryptoData,
};

What did we do?

Imported the getRequest function that we just made.
Created an async function named getCryptoData to fetch our data from https://www.cryptingup.com/api/markets.
Pass our data through the cryptoRender helper function we made earlier before returning the result.

Controllers

Now our api calls are sorted, we need to create a controller for our crypto data.

Let's create the controller using:

touch controllers/crypto.controllers.js

Now let's add the following to our controllers/crypto.controllers.js

const { shortCache } = require("../middleware/cache");
const { getCryptoData } = require("../services/crypto.services");

const getCryptoList = async (ctx) => {
  try {
    const resData = await getCryptoData();
    shortCache.set("crypto-list", resData);
    ctx.body = resData;
    ctx.status = 200;
  } catch (err) {
    ctx.body = "Error!";
    ctx.status = 500;
    console.log(err);
    throw err;
  }
};

module.exports = {
  getCryptoList,
};

So what is happening?

Here's what we're doing:

Imported our cache (shortCache) and our crypto service (getCryptoData) to our controller.
Created an async function to handle errors that may arise.
Called our getCryptoData function and await the response data.
Stored the response data in the cache under "cryto-list".
Return the response data.

The first request we make will take a couple seconds as we are interacting with an external api.

Once the cache sets our data under "crypto-list", for the given duration - our middleware will detect that "crypto-list" has data stored and will return that data rather than executing our controller.

Router

Finally let's update our router with the following code:

const Router = require("koa-router");
const router = new Router();
const { cryptoListCache } = require("./middleware/crypto.cache");
const { getCryptoList } = require("./controllers/crypto.controllers");
const { getEvents, postEvent } = require("./controllers/events.controllers");

router.get("/crypto_list", cryptoListCache, getCryptoList);
router.get("/events_list", getEvents);
router.post("/post_event", postEvent);

module.exports = router;