Hello everyone! I am Atibhi Agrawal, a final year student from IIIT-Bangalore pursuing my Integrated Masters in Technology. This blog is about my experience contributing to Prometheus as a part of my internship at Grafana Labs. It is not going to be a technical blog. The objective of this blog is to document my own journey as well as help others contribute to Prometheus or any large open source project.

Learning about Prometheus

Prometheus is an open-source systems monitoring and alerting toolkit. It has a query language PromQL and a powerful data model that allows us to monitor our applications and infrastructure. In recent times, it has become the de-facto standard for application monitoring in the cloud-native space.

I had never ventured into the monitoring domain or heard about Prometheus before this internship. To get started with it, my mentors David Kitchen and Björn "Beorn" Rabenstein suggested some videos and documents that would help me learn about Prometheus.

Here are a some helpful resources :

Prometheus: A Next Generation Monitoring System

Intro: Prometheus --- Matt Layher, Fastly & Ganesh Vernekar, Grafana Labs

Prometheus Deep Dive

Official Website

Awesome Prometheus

Apart from watching videos and reading, I suggest getting your hands dirty and setting up Prometheus to gain a better understanding. Prometheus basics by yolossn is a great tutorial that was very helpful for me. After I got familiar with Prometheus, I was assigned a task. My task was to Add Backfilling support for Prometheus. The codebase of Prometheus is in Go, and since I had no prior experience with Go, I started learning the language.

Learning Golang

Whenever I learn a new programming language, I start by building a small project or follow along with interactive tutorials/videos. While coding along, I almost always run into errors. Debugging and trying to fix those errors helps me explore the language more. I also document the errors or new concepts that I come across in any programming language so that the next time something similar comes up, I can refer to my notes rather than hunt for the solution on stack overflow.

To learn Go, here are some free resources that I used :

A Tour of Go

Learn Go Programming --- Golang Tutorial for Beginners

I also made use of the free Udemy subscription offered by Grafana Labs to its employees and completed a paid course.This course was very helpful.

My first Pull request to Prometheus

After learning more about the project and the language, I started the work. I first created a design document that I mailed to the prometheus-developers mailing list. A design document is a high-level solution to a problem, it generally contains the design decisions, trade-offs and an overview of the implementation. It is similar to the concept of RFC's.

After getting approval from the Prometheus community, I started coding. I opened a PR after getting the basic structure in place. This PR certainly started small but it slowly kept growing. Initially, a few changes were requested in the PR but as the PR progressed more and more feedback was given and more changes were requested. In the end the PR had 275+ discussions. A lot of them revolved around Go and coding best practices. My mentor Beorn also sent me this when the reviews were constantly coming in.

The code reviews taught me a lot and really helped me grow. Here are some of my key learnings :

• Keep PRs small.
• Add lots of unit tests, this helps in avoiding bugs in production.
• Learn about DRY software practice which stands for "Do not Repeat Yourself" and refers to avoiding code duplication.
• Code should be modular and readable.
• Searching for examples in the code base teaches the idiomatic way to do something quicker than other resources. Follow the conventions in the rest of the code base.

Finally after multiple iterations my first PR to Prometheus was merged. It was so satisfying 💃

First PR merged on Nov 26

I even got the opportunity to give a lightning talk about my work at ObservabilityCon.

The Prometheus community

Since I contributed to Prometheus as a part of my internship, I was lucky to have direct access to the Prometheus developers working at Grafana Labs whenever I had any questions. Initially, I was very hesitant to ask for help for fear of sounding stupid but my mentors Beorn, Ganesh and Goutham were very kind and encouraged me to ask questions. They answered my queries and provided me feedback that allowed me to improve.

We also set up a common time every week with the CNCF mentee Jess, who was working on something similar at the time, and Bartek, another Prometheus developer, to discuss anything that might be blocking us. They were also very kind and helpful and I enjoyed interacting with them. I really thank them for their time and help.

If you are an external contributor, there are many options as well, like the Prometheus contributor hours every Monday where anyone contributing to Prometheus can join in to ask for help when facing an issue. One can also hop on to IRC: #prometheus on irc.freenode.net (for the easiest start, join via Element) which is more for users. Developers can ask for help on #prometheus-dev on IRC. Most of the maintainers are friendly and encouraging.

Final Thoughts

I learnt a lot by contributing to Prometheus. It gave me an opportunity to learn about monitoring, distributed systems, Go, cloud-native technologies etc. Prometheus is a popular open-source project that takes part in the Google Summer of Code and Cloud Native Computing Foundation Mentorship program every year and I highly recommend you to contribute to the project. It will hopefully be a good learning experience for you too.

Feel free to comment if you need further guidance or DM me on Twitter. You can also reach me at atibhi.a@gmail.com 😄