Monitoring Flask with OpenTelemetry and Uptrace

Still using Jaeger/Sentry? Uptrace is an open-source APM tool that supports distributed tracing, metrics, and logs. You can use it to monitor applications and set up automatic alerts to receive notifications via email, Slack, Telegram, and more.

What is tracing?

Distributed Tracing allows you to follow a request through multiple services and components, capturing the timing of each operation as well as any logs and errors that occur.

In a distributed environment, tracing further enhances your understanding of the relationships and interactions between distributed microservices and systems.

Tracing allows you to break down requests into spans. A span is an operation (unit of work) that your application performs when handling a request, such as a database query or a network call.

A trace is a tree of spans that shows the path a request takes through an application. The root span is the first span in a trace.

What is OpenTelemetry?

OpenTelemetry is an open-source observability framework for OpenTelemetry tracing (including logs and errors) and OpenTelemetry metrics.

OpenTelemetry allows developers to instrument their applications to automatically generate telemetry data that can then be collected and analyzed to gain insight into the performance, behavior and health of the system. It supports multiple programming languages and integrates with multiple observability tools and platforms.

Otel allows developers to collect and export telemetry data in a vendor agnostic way. With OpenTelemetry, you can instrument your application once and then add or change vendors without changing the instrumentation, for example, here is a list of DataDog competitors that support OpenTelemetry.

Creating spans

To measure the performance of database queries or HTTP requests, you can create a span using the OpenTelemetry Python API:

from opentelemetry import trace

tracer = trace.get_tracer("app_or_package_name", "1.0.0")

def some_func(**kwargs):
    with tracer.start_as_current_span("some-func") as span:
        // the code you are measuring

To record contextual information, you can annotate spans with attributes. For example, an HTTP endpoint may have such attributes as http.method = GET and http.route = /projects/:id.

# To avoid expensive computations, check that span is recording
# before setting any attributes.
if span.is_recording():
    span.set_attribute("http.method", "GET")
    span.set_attribute("http.route", "/projects/:id")

You can also record exceptions and set the span status code to indicate an error:

except ValueError as exc:
    # Record the exception and update the span status.
    span.record_exception(exc)
    span.set_status(trace.Status(trace.StatusCode.ERROR, str(exc)))

See OpenTelemetry Python tracing for details.

What is Uptrace?

OpenTelemetry relies on a backend system to store, process, analyze, and visualize the telemetry data collected from applications. As a critical component, the backend enables you to extract valuable insights and make informed decisions based on the collected telemetry data.

Uptrace is an open source APM for OpenTelemetry with an intuitive query builder, rich dashboards, automatic alerts, and integrations for most languages and frameworks.

You can get started with Uptrace by downloading a DEB/RPM package or a pre-compiled Go binary.

Example application

In this tutorial, you will be instrumenting a toy app that uses Flask and SQLAlchemy database client. You can retrieve the source code with the following command:

git clone git@github.com:uptrace/uptrace.git
cd example/flask

The app comes with some dependencies that you can install with:

pip install -r requirements.txt

Configuring OpenTelemetry

Uptrace provides OpenTelemetry Python distro that configures OpenTelemetry SDK for you. To install the distro:

pip install uptrace

Then you need to initialize the distro whenever you app is started, for example, in manage.py:

# manage.py

import uptrace

def main():
    uptrace.configure_opentelemetry(
        # Copy DSN here or use UPTRACE_DSN env var.
        # dsn="",
        service_name="myservice",
        service_version="v1.0.0",
    )

    # other code

Instrumenting Flask app

To instrument Flask app, you need to install a corresponding
OpenTelemetry Flask instrumentation:

pip install opentelemetry-instrumentation-flask

Then you can instrument the Flask app:

from opentelemetry.instrumentation.flask import FlaskInstrumentor

app = Flask(__name__)
FlaskInstrumentor().instrument_app(app)

Instrumenting SQLAlchemy

To instrument SQLAlchemy database client, you need to install a corresponding OpenTelemetry SQLAlchemy:

pip install opentelemetry-instrumentation-sqlalchemy

Then instrument the db engine:

from flask_sqlalchemy import SQLAlchemy
from opentelemetry.instrumentation.sqlalchemy import SQLAlchemyInstrumentor

app.config["SQLALCHEMY_DATABASE_URI"] = "sqlite:///:memory:"
db = SQLAlchemy(app)
SQLAlchemyInstrumentor().instrument(engine=db.engine)

Running the example

You can start Uptrace backend with a single command using
Docker example:

docker-compose up -d

And then start the app passing the Uptrace DSN as an env variable:

export UPTRACE_DSN=http://project2_secret_token@localhost:14317/2
python3 main.py

The app should be serving requests on http://localhost:8000 and should render a link to Uptrace UI. After opening the link, you should see this:

What's next?

Next, instrument more operations to get a more detailed picture. Try to prioritize network calls, disk operations, database queries, error and logs.

You can also create your own instrumentations using
OpenTelemetry Python Tracing API.

Blog