In an investigation, assumption kills! - Jack Reacher

What happened?

The dashboard of the management system that I’m working on shows various metric information in widget type. There are some widgets that when you click on them, will show a pop-up that contains detailed information presented in the table.

For some unknown reason, the detail list shows less data than it should ( less than the number shown on the widget itself).

The investigation

• The investigation begins with one thought standing in my mind:
  The backend code has run just fine and it hasn't changed since 2023!!!

• With that mindset, I started to check the API call and server log,...
  The data of the detail list is from ElasticSearch service so I assumed that the error started because of the ES data.

• I started trying to build the ES query that the backend generated, I think when I got the query, I could get the data from ES service, check the result, and find out the problem. It took me two weeks!!!

• After trying and failing several times, I realized I had taken the wrong path. I checked the API call again and...it shining right in front of my face, just slap a big hand at me.
  So the error didn't come from ES data at all. The difference between the number in the widget and the total record in the detail table is the number in the widget could be the result of an expression but in the detail table, it's NOT.
  eg: the widget number = result from query A + result from query B
  the total record in detail table = result from query A

Result

• After realizing how dumb I was, I started to implement some code in the back-end code so the detail table would show the correct number of records as the widget when the expression is deactivated ( yes, this business logic is confirmed by the client)

Conclusion

• The mindset that I had during the early day of fixing the bug was a beginner mistake and I didn't realize it.
• Should change how I approach the problem sooner after the path leads to nowhere.
• Always keep looking at the evidence, keep a clear mind, and don't make assumptions.