The outbox pattern is a strategy aimed at solving potential external communication problems, such as a message broker. If the communication is not carried out, the consistency and atomicity of operations may not be guaranteed. Its implementation increases the reliability and robustness of communication between services, ensuring that messages are not lost in case of communication problems, such as network failures or the target service being down. The implementation of the outbox pattern aids in error recovery, since messages are persisted in the service that issued them.

How does the outbox pattern work?

Let's say we need to publish a message on rabbitmq, but for some reason, the service is unavailable. It would be impossible to publish this message, even if the application was resilient and tried several times. This would leave the system in an unaware state.

To solve this, whenever it is necessary to communicate with an external service, the message to be sent should be persisted in the database as part of the local transaction. The OUTBOX table acts as a temporary message queue. A separate process, usually a background job, is responsible for reading the local queue and sending the messages to the external service. This process will continue to attempt to publish the message until the service is available and will not lose the message. In this way, atomicity is guaranteed because it involves a local ACID transaction.

Source: Chris Richardson, in the book Microservice Patterns.

Do I always need to implement the outbox pattern?

All approaches in software architecture have their pros and cons, so the answer is: "it depends!" 🤔. The implementation of the outbox pattern introduces complexity into the code. To answer this question, we must ask ourselves whether it is necessary to maintain consistency and atomicity in the transaction. The outbox pattern may not be useful in certain cases, such as sending a non-critical notification to the user. If the occasional loss of data is not important, it is not worth implementing the outbox pattern. In cases where the potential loss of data is critical and it is necessary to maintain atomicity and consistency across multiple services, implementing the outbox pattern is an added value for the reasons already mentioned.

Implementation in .net:

The following image presents a basic example of how to implement the outbox pattern in C#. In this example, the creation of a category and the registration of the same in a queue are implemented. At the moment of persisting the new category, the event of its creation is also persisted.
Finally, a background service is created and registered that reads the oldest record and will try to place it in the queue. Note that this is just an illustration.

To ease the implementation and management of queues, tools such as Hangfire or Quartz can be used. Hangfire has the advantage of having a very appealing UI that allows for an overview of the state of the queues. Moreover, it also allows for manual management of them. In the following image, the same example is presented, implementing Hangfire.