Serverless error-handling pipeline using Redis
Rayan Dasoriya
Posted on September 5, 2020
Errors are an essential part of an application. These are the thing that you don't want to occur, but if they do, then there should be some mechanism to detect them. These errors can be due to some service unavailability, bugs, or any other unforeseen issue. In this article, I am going to highlight how we used IBM Cloud Functions, a serverless offering by IBM, and Redis to log the errors in our Slack channel.
Background
We have a Node.js application that interacts with Salesforce CRM offerings and performs certain operations. There are times when the application gets an error message. Whenever we get these messages, we want to detect and log these errors into our Slack channel so that appropriate actions can be taken.
Technology used
We used IBM Cloud Functions, which is a FaaS(Functions as a Service) platform based on Apache Openwhish. It allows us to run the application without worrying about the configurations. Redis cache was used to store the error logs within our system. It is an in-memory key-value data structure with in-build replication and LRU cache features.
Working
Our Node.js application runs within our IBM Cloud Kubernetes cluster which interacts with Salesforce CRM systems. Whenever it sees an error, we trigger IBM Cloud Functions. At this point, the Node.js application continues its work in the Kubernetes cluster without worrying about the errors. The control for error handling is being transferred to IBM Cloud functions.
IBM Cloud functions try to establish a connection with the Redis cluster and stores the log over there. We categorize the type of error and then check if the error is eligible to be sent over to the Slack channel. If we have to send the message on Slack, we use Slack Webhooks. Once this message is sent, we remove the log from Redis.
We are using Redis to make sure that our logs have been processed. If we get an error next time, we will check if our Redis cluster contains any error messages that need to be processed. It will then retry sending these messages to Slack, thus helping us to ensure we don't miss any errors.
Posted on September 5, 2020
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.