Serverless error-handling pipeline using Redis

rayandasoriya

Rayan Dasoriya

Posted on September 5, 2020

Serverless error-handling pipeline using Redis

Errors are an essential part of an application. These are the thing that you don't want to occur, but if they do, then there should be some mechanism to detect them. These errors can be due to some service unavailability, bugs, or any other unforeseen issue. In this article, I am going to highlight how we used IBM Cloud Functions, a serverless offering by IBM, and Redis to log the errors in our Slack channel.

Background

We have a Node.js application that interacts with Salesforce CRM offerings and performs certain operations. There are times when the application gets an error message. Whenever we get these messages, we want to detect and log these errors into our Slack channel so that appropriate actions can be taken.

Technology used

We used IBM Cloud Functions, which is a FaaS(Functions as a Service) platform based on Apache Openwhish. It allows us to run the application without worrying about the configurations. Redis cache was used to store the error logs within our system. It is an in-memory key-value data structure with in-build replication and LRU cache features.

Working

Our Node.js application runs within our IBM Cloud Kubernetes cluster which interacts with Salesforce CRM systems. Whenever it sees an error, we trigger IBM Cloud Functions. At this point, the Node.js application continues its work in the Kubernetes cluster without worrying about the errors. The control for error handling is being transferred to IBM Cloud functions.
Application Workflow
IBM Cloud functions try to establish a connection with the Redis cluster and stores the log over there. We categorize the type of error and then check if the error is eligible to be sent over to the Slack channel. If we have to send the message on Slack, we use Slack Webhooks. Once this message is sent, we remove the log from Redis.

We are using Redis to make sure that our logs have been processed. If we get an error next time, we will check if our Redis cluster contains any error messages that need to be processed. It will then retry sending these messages to Slack, thus helping us to ensure we don't miss any errors.

💖 💪 🙅 🚩
rayandasoriya
Rayan Dasoriya

Posted on September 5, 2020

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related