Introduction

In this series, we've explored how to get started with Shopify webhooks, and shared step-by-step tutorials on how to create webhooks using both the Shopify admin dashboard and Shopify API. Now, we'll conclude the series by looking at some important best practices for building resilience and reliability into Shopify webhooks in a production environment. We want reliable Shopify webhooks because sudden spikes in your Shopify webhook traffic can overload your API and cause it to shut down if the capacity of your API server is exhausted. Shopify can also send duplicate webhooks that could potentially cause inconsistencies in your application database.

In this article, we will share and suggest some key engineering practices that will help make your Shopify webhooks resilient to the usage pressures common in production environments.

Shopify webhooks features

Before getting into best practices for Shopify webhooks, let's take a look at its features. Understanding the strengths and weaknesses of Shopify webhooks and how they operate helps engineers design the most efficient solutions to mitigate any issues that might arise when using them.

Now that you're familiar with the features of Shopify webhooks, let's go ahead and break down solutions to issues that may arise in production environments.

Shopify webhooks best practices checklist

Local troubleshooting and testing

Properly testing your Shopify webhooks helps you avoid all the performance issues that arise due to buggy code. For example, let's say your endpoint on an external accounting application assumes that Shopify will send the total price for a store order as an integer value (e.g. 10). Meanwhile, Shopify sends the price as a floating-point number (10.15). Your accounting application then forcefully coerces the floating number into an integer value which does not reflect the true cost of the order, and will lead to inconsistencies in the accounting data.

You need to test your Shopify webhook endpoint to ensure the following:

• The endpoint exists (to avoid 404 errors)
• The endpoint does not perform a redirect (Shopify sees a redirect as an error)
• The logic on the endpoint is bug-free and has the desired impact on your application
• You return a proper status code for successfully processing the webhook (2xx range of HTTP status code) and for failure (4xx, 5xx range of HTTP status codes)

One drawback in testing Shopify webhooks is that webhooks cannot be received in a local development environment because they require a publicly accessible URL. Fortunately, tools like the Hookdeck CLI exist to solve this problem by tunneling your Shopify webhook requests to your local environment.

Asynchronous processing to ensure reliability

When Shopify sends a webhook, it waits for a response from the receiving application. This waiting period elapses after 5 seconds. Thus, if Shopify does not receive a response within the waiting period, Shopify assumes that the webhook delivery has failed.

As a best practice, you need to respond to Shopify as fast as possible. This practice is recommended on Shopify's webhook documentation. That said, it might not be possible to respond so quickly under certain conditions, such as:

1) Long-running processes

Your webhook can trigger a time-consuming task, such as file encryption, database queries, or computations that take long periods to complete. It may be difficult, and sometimes impossible, to send a response within the 5-second limit in this situation.

2) Multiple webhooks

Let's assume that the time it takes your application to process a single webhook is 3 seconds; you would be able to return a response within the 5-second limit. However, if your application receives 3 webhooks at the same time, the time required becomes 9 seconds and the second webhook will not complete before it times out.

Imagine you have a Shopify store that sells flowers, and you process an average of 10 orders daily. You have set up a webhook that targets an external delivery service when a customer places an order. Your delivery service comfortably handles your daily average load and then suddenly, Valentine's Day season rolls by, and the number of orders for flowers on your store surges to an all-time high of 300 orders per day.

You start receiving 300 webhooks daily on your delivery service, which causes the server resources for the application to be used up quickly and drop webhooks due to service failure. As webhooks begin to drop, many customers that have ordered and paid for their flowers won't have their flowers delivered.

So, how do we mitigate these often unavoidable situations? One of the standard solutions is to boost processing speed by expanding the server hardware capabilities. You can also adopt horizontal scaling techniques to distribute your webhook workload.

The above solutions are good, but the best way to respond to Shopify as quickly as possible is by processing the webhooks asynchronously.

With asynchronous processing, Shopify does not need to wait for your application to finish processing a webhook before it receives a response.

One of the ways to implement asynchronous processing for Shopify webhooks is by placing a message broker between Shopify and your application. The message broker ingests webhooks from Shopify and immediately sends a response back to Shopify. The broker then routes the webhooks to your application at a rate that your application can handle.

This way, your Shopify webhooks never time out, and your application does not run out of server resources.

Verifying your incoming webhooks

The next best practice we'll look at is security. The HTTPS endpoint used by your applications to receive Shopify webhooks is publicly accessible, which means that any client can call it.

This open-ended nature of your webhook URL is a huge security flaw, as attackers can spoof webhook requests to your application for various malicious purposes. You need a way to prevent your webhook endpoints from receiving requests that are not from Shopify.

Let's imagine that you have a Shopify store that sells basketball jerseys, with a webhook set up to message a delivery service about the order details for the jersey to be delivered to the customer's address. Without verification, an attacker can spoof order webhooks to your delivery service webhook URL and get free jerseys anytime they want.

Fortunately, Shopify provides a way to help us with this. Shopify sends a hashed version of the body of the payload in the X-Shopify-Hmac-Sha256 header (if you're using Ruby on Rails or Sinatra, the header is HTTP_X_SHOPIFY_HMAC_SHA256 ). The value of this header is a base64 encoded string and serves as the digital signature for the webhook payload. This signature is signed using your Shopify app's shared secret. For more information about Shopify app secrets and how to generate one, check the docs page.

To verify the payload on your endpoint handler, you need to re-compute the signature using your Shopify secret, the webhook payload, and some HMAC algorithm functions. Once you have your version of the payload signature, you can then compare it with the one sent in the X-Shopify-Hmac-Sha256 to confirm a match.

Below is an example of the verification process written in Node.js:

const getRawBody = require("raw-body"); //install raw-body from npm
const crypto = require("crypto");
const secret = MY_SHOPIFY_SECRET;

app.post("/webhook", async (req, res) => {
//Extract X-Shopify-Hmac-Sha256 Header from the request
  const hmacHeader = req.get("X-Shopify-Hmac-Sha256");

//Parse the request Body
  const body = await getRawBody(req);
//Create a hash based on the parsed body
  const hash = crypto
    .createHmac("sha256", secret)
    .update(body, "utf8", "hex")
    .digest("base64");

  // Compare the created hash with the value of the X-Shopify-Hmac-Sha256 Header
  if (hash === hmacHeader) {
    console.log("Webhook source confirmed. Continue processing");
    res.sendStatus(200);
  } else {
    console.log("Unidentified webhook source. Do not process");
    res.sendStatus(403);
  }
});

It is also worth mentioning that you should store your Shopify secret in an environment variable, especially in production environments. The entire process of verifying your webhook breaks down if an attacker can get hold of your Shopify secret.

Idempotency in webhook processing

Idempotency helps prevent the negative impact of processing a webhook more than once. Let's imagine that we need to deduct the price of a purchased item from a customer's wallet (the wallet application is external to the Shopify store). A charge is deducted from the wallet when we receive a webhook confirming the purchase. If this process is done more than once for a single purchase, the customer's wallet is now in a state that is inconsistent with their purchase history.

Situations like this negatively impact the integrity of your application's data, and in extension, the Shopify store itself.

While some activities, like updating the status of an order, are by nature idempotent, others like the one described above are not. You need a mechanism built into your webhook processing operations to ensure that no webhook is processed more than once.

Shopify helps to achieve this by sending the unique identifier for a webhook in the X-Shopify-Webhook-Id. With this identifier, you can track webhooks that have already been processed and skip them to avoid duplicating their impact on your application.

Dealing with delayed webhooks

On rare occasions, Shopify might send a webhook late (sometimes up to a day late). If the data in the payload is very critical to your application, you should verify how recent the data is. Stale data has the potential to cause state inconsistencies within your application, thereby compromising the database.

For example, think of an external inventory system where the quantity of an item is updated (i.e. deducted) after a successful purchase on Shopify. If the item is no longer available (quantity = 0 in the inventory system) and you have a late purchase webhook coming in, the current quantity in the inventory will either be negative, undefined, or default to zero based on the business logic implemented. However, this new inventory value does not reflect the actual state of the item quantity.

Shopify does its best to send the most recent data with a webhook even when the webhook delays. As an engineer, you know that assumptions can easily lead to bugs, so you want to make sure you're verifying how recent the webhook data is despite the promise by Shopify.

One way to implement this verification is by comparing the timestamp of the webhook to the current time. If the timestamp on the webhook is behind the current time, you can call the Shopify API to request the current data. This check ensures that you're getting the most up-to-date data for the Shopify resource.

Implementing reconciliation jobs

This best practice falls under the housekeeping category. Webhooks are sent to notify your external application and send real-time data about an event on a Shopify resource. However, Shopify advises that you shouldn't solely rely on webhooks because, and this is very rare, the webhook for a particular event might not be sent.

An example of a scenario where a reconciliation job is required would be if you had a Shopify store selling shirts and an external accounting application integrated with your store using webhooks.

Let's assume that you need to balance your accounts in the external accounting application at the end of each business day. A reconciliation job can be scheduled to run 30 minutes before you query your transaction history to compute your balances. This job can be used to pull all purchase information from your Shopify store and synchronize it with your accounting application so that you have all the data you need up-to-date.

Thus, to mitigate the issue of unsent Shopify webhooks, you should implement reconciliation jobs to synchronize your application data with Shopify. These reconciliation jobs are scheduled tasks that periodically fetch data from Shopify for specific resources.

Reconciliation jobs should be set up for sensitive data within your application; these are data that need to be up-to-date at certain periods while your application is running.

Conclusion

Building reliability and resilience into your infrastructure ensures that your setup can withstand the pressures of production environments. In this article, we have looked at some best practices to follow to ensure that our Shopify webhooks function without issues. This list is not set in stone, and you're encouraged to take more precautions that help prevent your application integration with Shopify from falling short of expectations.

Happy coding!