Some people think that notifications that everything went well generate unnecessary “noise” in the communication.

On the other hand, some people see it as, "Let's celebrate the small victories!" 🎉

I invite you to look at this topic from another perspective, based on personal experiences in different real-world projects.

Have you ever found yourself in a situation where there were more failure notifications than passing tests?

Or, have you ever been “lucky” to work on a project where red pipelines were an exception?

I've lived both situations.

In the one where tests failed often, I had the following impressions. 1. It seemed that the application being developed was always broken (that people didn't care about quality); 2. it “smelled” of non-deterministic, and therefore unreliable, automated tests; and 3. triaging failures was “hell in earth” (unexpected instabilities often caused failures).

In the second experience, though, it was exactly the opposite, and my perceptions were. 1. The default was green pipelines ✅ , that is, quality was considered a first-class citizen; 2. when there was a failure ❌, soon the team self-organized to fix it and the pipeline turned green again ✅; and 3. when something has gone wrong ❌, we could look at all the other times it went right before it broke, alleviate any feelings of guilt, and refocus on quality through bug-fixing. ✅

Okay, but after all, do I only notify failures or all executions?

One approach that works is to have two channels, one for failures and one with all the execution history. The point is, remember to look at this history from time to time to understand the “healthy” of your project during its life. If the pattern is passing pipelines, you might be fine. Keep it up! If the pattern is more failures than green pipelines, stay alert ⚠️ and solve the problem's root cause.

What do you think of all that? Please, leave a comment.

This post was originally published in Portuguese on the Talking About Testing blog.

👋 Until next time and happy testing!