How Deliveroo used a Redux middleware to wrap their http client

Readers of this article should be confortable with Async data flows. Knowledge of Redux core concepts such as State, Actions and Reducers is a plus but principles applied here are relevant for any http client one might build.

Today we are going to talk about how Deliveroo used a Redux middleware to structure their API Client layer through carefully designed actions.

After a brief introduction about Redux middlewares, we will dive right into the matter with a step-by-step analysis of how Deliveroo built their API Redux middleware.

Redux Middlewares

Middlewares are not specific to Redux. For instance, the Express framework can be considered as a stack of middleware functions. Those functions sit in the middle of the request/response cycle, performing operations such as logging or altering response headers.

According to the Redux doc about middlewares:

Redux middleware provides a third-party extension point between dispatching an action, and the moment it reaches the reducer. People use Redux middleware for logging, crash reporting, talking to an asynchronous API, routing, and more.

One of the most common middlewares is Redux Thunk, which allows one to dispatch async actions:

// https://github.com/reduxjs/redux-thunk/blob/master/src/index.js

const thunkMiddleware = ({ dispatch, getState }) => (next) => (action) => {
  if (typeof action === 'function') {
    return action(dispatch, getState, extraArgument);
  }

  return next(action);
};

As you can see, it's pretty simple: if the action passed to dispatch is a function, it calls the function, otherwise it just passes the action down the middlewares pipeline by returning next(action). It is an handy way to intercept an action on its way to the reducer, and perform some logic based on its type.

The ({ dispatch, getState }) => (next) => (action) => { ... } syntax might seem odd, but it is really only three nested function calls using arrow functions. It can be rewritten as:

function thunkMiddleware({ dispatch, getState }) {
  return function wrapDispatch(next) {
    return function handleAction(action) {
      if (typeof action === 'function') {
        return action(dispatch, getState, extraArgument);
      }

      return next(action);
    }
  }
};

If you want to learn more about Redux middlewares, the Redux documentation has a great section about the logic behind its implementation.

Deliveroo API Middleware

Deliveroo is a food delivery startup from the UK. Their frontend app, as of July 2019, was a React universal app, built with NextJS and Redux. The code featured in this part was extracted using sourcemaps. Unfortunately sourcemaps are not available on Deliveroo anymore. Thus this code reflects the state of the app back in July 2019. It may be different today.

Deliveroo used a Redux middleware to wrap their API client : every action with a specific type is being picked up by the middleware, which takes care of requesting the API, normalizing the response, and dispatching appropriate SUCCESS or FAILURE actions depending on the result of the API call.

Server considerations have been removed from the code snippets below, for simplicity sake, as it is beyond ths scope of this post. Without further ado, let's dive into Deliveroo's code and get the key takeaways from their middleware implementation.

Intercepting action calls to the API

Let's start with the specification Deliveroo engineers wrote for this middleware:

/*
  A middleware for making requests to the deliveroo API
  =====================================================

  Any action that returns an object that has the following shape will be
  picked up by this function

  {
    type: 'LOGIN',
    endpoint: '/orderapp/v1/login', // the hook for this middleware.
    authorized: false,              // send autorization headers?
    options: {
      ...fetchOptions,
    },
    onSuccess: (res) => undefined,
    onFailure: (res) => undefined,
  }

*/

The prerequesite for such an action to be picked up is to have an endpoint key. This translates into code as:

// utils/requestHelper.js

const api = (store) => (next) => (action) => {
  // If the object doesn't have an endpoint, pass it off.
  if (!action.endpoint) return next(action);
}

If the action object endpoint key is undefined, we return the next middleware call using return next(action)

Request options

The action architecture allows for some custom options to be passed on to the incoming API request. These options, along with default options and configuration available in the Redux store are merged together to form the request options passed to the fetch call.

// middleware/api.js

var JSON_MEDIA_TYPE = 'application/json';
var JSONAPI_MEDIA_TYPE = 'application/vnd.api+json';
var defaultOptions = {
  headers: {
    'content-type': JSON_MEDIA_TYPE,
    accept: [JSON_MEDIA_TYPE, JSONAPI_MEDIA_TYPE].join(', ')
  },
  credentials: 'omit',
  // Set an aggressive default timeout, to prevent outbound calls queueing
  timeout: 5000
};

const api = (store) => (next) => (action) => {
  if (!action.endpoint) return next(action);

  // Building the request options
  const options = {};
  const { request, config } = store.getState();
  const requestOptions = {
    headers: buildHeadersFromRequest({ request, config })
  };

  defaultsDeep(options, action.options, requestOptions, defaultOptions);

  next({ type: `${action.type}_REQUEST` });

  // Default to the orderweb API host unless an action overrides
  const host = action.host || configUtil.orderappApiHost;

  if (!host) {
    throw new Error('Unable to find valid API host for fetch');
  }
  const url = `${host}${action.endpoint}`;
}

The buildHeadersFromRequest function gives us some information on the request-related data stored in the Redux store:

// utils/requestHelper.js

export const buildHeadersFromRequest = ({ request, config = {} }) => {
  const {
    apiAuth,
    country,
    currentUrl,
    ip,
    locale,
    referer,
    rooGuid,
    rooSessionGuid,
    rooStickyGuid,
    userAgent,
  } = request;

  const authorizationHeader = (requestApiAuth) => {
    if (!requestApiAuth) {
      return '';
    }
    if (requestApiAuth.indexOf('.') !== -1) {
      // Only JWT based API Auth will have a period in it
      return `Bearer ${requestApiAuth}`;
    }
    // Opaque-token based authentication with Orderweb
    return `Basic ${requestApiAuth}`;
  };

  /*
    Use the sticky guid from
      - The cookie in the request if present.
      - From config if a cookie isn't set.
    If neither option has a stickyguid fallback to the users normal guid.
  */
  const stickyGuid = rooStickyGuid || config.rooStickyGuid || rooGuid;

  return Object.assign(
    {},
    {
      'Accept-Language': locale,
      Authorization: authorizationHeader(apiAuth),
      'User-Agent': `${userAgent} (deliveroo/consumer-web-app; browser)`,
      'X-Roo-Client': 'consumer-web-app',
      'X-Roo-Client-Referer': referer || '',
      'X-Roo-Country': country.tld,
      'X-Roo-Guid': rooGuid,
      'X-Roo-Session-Guid': rooSessionGuid,
      'X-Roo-Sticky-Guid': stickyGuid,
    },
  );
};

Those headers are mainly related to locales, Authorization and tracking.

Making the request

Once everything is set up, the API call is made using fetch:

// middleware/api.js

const api = (store) => (next) => (action) => {
  // ACTION INTERCEPTION

  // OPTIONS SETUP

  return fetch(url, options)
    .then(response) => {
      // RESPONSE HANDLING
    }
}

Handling the response

The call itself is not very insightful, however the response handling is way more interesting. Let's first start with the "unhappy path", where the response is not 200 OK:

// middleware/api.js

const api = (store) => (next) => (action) => {
  // ACTION INTERCEPTION

  // OPTIONS SETUP

  return fetch(url, options)
    .then((response) => {
      if (!response.ok) {
        // If the response is not okay and we don't recieve json content
        // return data as undefined.
        const contentType = response.headers.get('content-type');
        const contentLength = response.headers.get('content-length');

        if (contentLength === '0') {
          // eslint-disable-next-line prefer-promise-reject-errors
          return Promise.reject({
            data: { message: response.statusText },
            status: response.status,
          });
        }

        if (contentType && contentType.indexOf(JSON_MEDIA_TYPE) !== -1) {
          return response
            .json()
            .catch(
              // eslint-disable-next-line prefer-promise-reject-errors
              (err) => Promise.reject({ data: err, status: response.status }),
            )
            .then(
              // eslint-disable-next-line prefer-promise-reject-errors
              (data) => Promise.reject({ data, status: response.status }),
            );
        }

        // eslint-disable-next-line prefer-promise-reject-errors
        return Promise.reject({ data: undefined, status: response.status });
      }
}

If the response is not OK, a rejected Promise object is returned. The data in the object depends on the response from the API. When JSON data is present in the response, we parse it and add it to the rejected Promise object. This method permits catching failed requests in catch directly from the fetch call. A createExceptionHandler method handles the error by either throwing, if the error is an instance of Error (this can happen if .json() fails for instance) or by dispatching a failure action that will be handled by the Redux reducer down the pipe.

// middleware/api.js

export const createExceptionHandler = (next, action) => (error) => {
  const isError = error instanceof Error;

  if (isError) {
    throw error;
  }

  let status = error.status || 500;
  const data = error.data || {};

  next({
    type: `${action.type}_FAILURE`,
    status,
    message: data.message || error.message,
    payload: {
      ...data,
    },
  });

  if (action.onFailure) action.onFailure(data);
};

const api = (store) => (next) => (action) => {
  // ACTION INTERCEPTION

  // OPTIONS SETUP

  return fetch(url, options)
    .then(response) => {
      if (!response.ok) {
        // Promis.reject(...)
      }
    }
    .catch(createExceptionHandler(next, action))
}

The "happy path" is handled in a similar fashion:

// middleware/api.js

export const JSONResponseHandler = (response, action) => (data) => {
  let parsedData;
  try {
    parsedData = JSON.parse(data);
  } catch (error) {
    // If the JSON fails to parse report an error to Sentry and add some
    // additional context for debugging. Then return a promise rejection.
    const err = new Error(
      `API Middleware - Browser: Failed To Parse JSON`,
    );

    return Promise.reject(err);
  }

  if (!parsedData) {
    // If the JSON successfully parses but the data is a falsey value,
    // i.e null, undefined, empty string.
    // Report the error to Sentry and return a promise rejection as
    // these values are likely to crash in the Reducers.
    const err = new Error(
      `API Middleware - Browser: Invalid JSON Response`,
    );
    Sentry.withScope((scope) => {
      scope.setExtras({
        action: action.type,
        status: response.status,
        data,
      });

      captureException(err);
    });
    return Promise.reject(err);
  }

  // If the JSON parses successfully and there is a body of data then return
  // the following block.
  return {
    payload: { ...parsedData },
    status: response.status,
    headers: response.headers,
  };
};

const api = (store) => (next) => (action) => {
  // ACTION INTERCEPTION

  // OPTIONS SETUP

  return fetch(url, options)
    .then(response) => {
      if (!response.ok) {
        // Promis.reject(...)
      }
    }

    if (response.status === 204) {
        return {
          payload: {},
          status: response.status,
          headers: response.headers,
        };
      }

      return response.text().then(JSONResponseHandler(response, action));
    }
    .catch(createExceptionHandler(next, action))
}

If the server returns a 204 No Content, a simple object with empty payload is returned, otherwise, the response is passed to the JSONResponseHandler, which in turn parses the JSON data and handles parsing errors. An object with the response headers, status as well as the parsed data as its payload is returned.

As one can see, response handling is quite complex, as many cases and errors can arise. Complexity is reduced here by using external functions to handle responses and execeptions. Rejecting a promise when errors surface, permits a global error handler in createExceptionHandler.

Bringing it home

The heavy duty work is behind us. After successfully handling the response, some data processing is needed (data denormalization, flattening..) before passing it down the middleware pipeline. This data processing is purely taylored to Deliveroo's needs in its actions and is not relevant to dig into here (by inspecting the jsonApiParser):

// midlleware/api.js

const api = (store) => (next) => (action) => {
  // ACTION INTERCEPTION

  // OPTIONS SETUP

  return fetch(url, options)
    .then(response) => {
      if (!response.ok) {
        // Promis.reject(...)
      }

      return response.text().then(JSONResponseHandler(response, action));
    }
    .then((response) => {
      const contentType = response.headers.get('content-type');
      if (contentType === JSONAPI_MEDIA_TYPE) {
        return {
          ...response,
          payload: jsonApiParser(response.payload),
        };
      }

      return response;
    })
    .catch(createExceptionHandler(next, action))
}

Once the data is taylored to our needs, we can move to the final step:

// middleware/api.js

const api = (store) => (next) => (action) => {
  // ACTION INTERCEPTION

  // OPTIONS SETUP

  return fetch(url, options)
    .then(response) => {
      if (!response.ok) {
        // Promis.reject(...)
      }

      return response.text().then(JSONResponseHandler(response, action));
    }
    .then((response) => {
      // DATA PROCESSING
     })
    .then((response) => {
      const requestKeys = action.payload ? Object.keys(action.payload) : [];
      const responseKeys = response.payload ? Object.keys(response.payload) : [];

      requestKeys.filter((key) => responseKeys.indexOf(key) !== -1).forEach((key) =>
        // eslint-disable-next-line no-console
        console.warn(`API middleware: clashing keys in the payload field. Overriding: ${key}`),
      );

      const newAction = {
        type: `${action.type}_SUCCESS`,
        status: response.status,
        payload: {
          ...action.payload,
          ...response.payload,
        },
        meta: {
          apiMiddleware: action,
        },
      };

      next(newAction);

      if (action.onSuccess) action.onSuccess(newAction);
    }

If request and response keys are clashing, a message is logged to the console, for debugging purposes, and probably tracking in Sentry. Finally, the SUCCESS Redux action is built using all the data from the previous steps: Response status, action and response payloads as well as metadata. The action is passed down the middleware stack using next(newAction). The action object has a onSuccess callback function to perform some custom behavior on a per-action basis.

Real-world action

To put what we just analyzed into perspective, what's better than a real-world example taken from Deliveroo's Client?

// actions/orderActions.js

export function getOrderHistory() {
  return (dispatch, getState) => {
    const { unverifiedUserId } = getState().request;

    const currentPageIndex = getState().order.history.orderHistoryPage;
    const pageOffset = ORDERS_PER_ORDER_HISTORY_PAGE * currentPageIndex;

    if (unverifiedUserId) {
      return dispatch({
        type: ORDER_HISTORY,
        /* TODO: Remove + 1 from ORDERS_PER_ORDER_HISTORY_PAGE once we get
        proper pagination from API */
        endpoint: `/orderapp/v1/users/${unverifiedUserId}/orders?limit=${ORDERS_PER_ORDER_HISTORY_PAGE +
          1}&offset=${pageOffset}`,
        payload: {
          /*
            TODO: This is to allow dummy data. This is not on page load,
            but only after clicking load more history
          */
          clickAndCollectOn: isFeatureActive(getState(), OH_CLICK_AND_COLLECT),
        },
        onSuccess: (response) => {
          /* TODO: Remove once we get proper pagination from API */
          if (response.payload.orders.length <= ORDERS_PER_ORDER_HISTORY_PAGE) {
            dispatch(hideLoadMoreOrderHistoryButton());
          }
        },
      });
    }

    return Promise.resolve();
  };
}

Here's an action to get the orders history for a user. One can notice the use of the onSuccess function to dispatch a "hide-button" action depending on the length of the orders.

Takeaways

In this article, we discovered how Deliveroo engineers implemented a Redux middleware to wrap their API client. It allows to avoid duplication of logic between different actions and offers a standardized way to communicate with the API, as well as a standardized response one can expect from it, in a least surprise way.

The middleware handles pretty much any response and any error that can occur in the lifecycle of the request. Even more, carefully implemented instrumentation, using Sentry, allows engineer to debug unexpected behavior efficiently.

This is a great demonstration of an http client implementation and of Redux middleware capabilities.