I have been working lately to implement some techniques regarding fault tolerance and chaos control in a microservice architecture.

I got inspired to write this article after implementing these techniques and reading more about how Netflix utilizes these patterns as well.

Circuit Breakers

It is a design pattern in the microservices world. It acts as a health check and availability controller.

Assuming a system that has a lot of microservices involved.
Microservice A calls Microservice B and for some reason Service B fails to respond due to network issues, logic faults, or availability.

The client might keep requesting a particular action since it has no idea that Microservice B will not be able to fulfill this action. As a result, this will cause low performance to the whole system, provide a bad user experience, and it might also cascade the issue to other healthy microservices.

The circuit breaker controls the flow of requests to the damaged microservice.

It has 3 states:

• Closed (Allowing requests to pass through).
• Open (Preventing the flow of requests).
• Half-Open (Allowing a limited number of requests to pass through)

The behavior is very similar to how electric current flows in a circuit.

In the closed state, the circuit breaker allows the flow of the requests to the microservices. At this state, the circuit breaker counts the amount of failed requests. When it reaches a specified threshold, the circuit breaker will change the state of the circuit to Open.

In the open state, the circuit breaker prevents the flow of the requests to the microservice, sending back a proper error message to the client. After that state, the circuit breaker will change the state of the circuit to Half-open depending on the Half-open time buffer.

In the Half-open state, the circuit breaker allows a limited number of requests to pass through. At the same time, it keeps track of how many of them are succeeding and failing.

If the number of succeeded requests reaches a specific threshold, the Circuit breaker will close the circuit again to ensure the flow of the requests to the microservices.

There are some open-source frameworks that support the implementation of circuit breakers for nodeJs that I find very helpful.

• opossum
• brakes
• levee

Conclusion

Using Circuit breakers is something that should be considered to help improve the fault tolerance of a microservice architecture. It also provides reliability and prevents the impacted damage to other microservices.