Overview of My Submission

Schedule metric collectors to gather measurements from different external services and optionally run anomaly detection on the new values.

The project uses MongoDB as a storage system, storing data as time series entries.

To make the experience complete, a visualization tool is needed. Fortunately, Mongo gives us Mongo Charts which can be used directly from Mongo Atlas.

Submission Category:

Prime Time

Link to Code

The code can be found here:

$ docker-compose build
$ docker-compose up

Additional Info

To test the project, I created three metrics, two of them are randomly generated (as their name suggests) and another one is the number of Dev.to posts in the last 15 minutes (I chose 15 minutes just so I don't have to wait so long for the data).

In the first screenshot, we can see the number of posts in the last 15 minutes and anther graph, plotting the values returned by random.randrange(1, 100)

In the second screenshot, we can see the second randomly generated metric (on the right) and a plot of the anomaly prediction (on the left). It is considered an anomaly if the score is -1.

As you can see, due to the metric's randomness and few data points, the anomaly prediction is not giving us any real insight.

Built with

Flask: Flask is a lightweight WSGI web application framework

MongoDB: An open-source NoSQL database developed for high performance, high availability, and easy scalability. It also provides Time Series collections.

Celery: A task queue with focus on real-time processing, while also supporting task scheduling.

Docker: An open platform for developing, shipping, and running applications

scikit-learn: Simple and efficient tools for predictive data analysis