Monitoring Express.js with OpenTelemetry and Uptrace

What is tracing?

In modern applications, especially those based on microservices or serverless architectures, different services often interact with each other to fulfill a single user request. This makes it challenging to identify performance bottlenecks, diagnose issues, and analyze the overall system behavior.

Distributed tracing aims to address these challenges by creating a trace, which is a representation of a single user request's journey through the various services and components. Each trace consists of a series of interconnected spans, where each span represents an individual operation or activity within a specific service or component.

What is OpenTelemetry?

OpenTelemetry is an open source observability framework hosted by Cloud Native Computing Foundation. It is a merger of OpenCensus and OpenTracing projects.

OpenTelemetry aims to provide a single standard across all types of observability signals such as OpenTemetry logs, distributed tracing, and metrics.

OpenTelemetry specifies how to collect and send telemetry data to backend platforms. With OpenTelemetry, you can instrument your application once and then add or change vendors without changing the instrumentation.

Creating spans

To start creating spans, you need a tracer. You can create a tracer by providing the name and version of the library/application doing the instrumentation:

const otel = require('@opentelemetry/api')

const tracer = otel.trace.getTracer('app_or_package_name', '1.0.0')

Once you have a tracer, creating spans is easy:

// Create a span with name "operation-name" and kind="server".
tracer.startActiveSpan('operation-name', { kind: otel.SpanKind.SERVER }, (span) => {
  doSomeWork()
  span.end()
})

OpenTelemetry Node.js

Using the OpenTelemetry Node.js library, you can easily add distributed tracing capabilities to your Express.js applications.

To instrument an Express.js app, you need to install OpenTelemetry Node.js SDK and available instrumentations:

# Using npm
npm install @opentelemetry/sdk-node @opentelemetry/api @opentelemetry/auto-instrumentations-node

# Using yarn
yarn add @opentelemetry/sdk-node @opentelemetry/api @opentelemetry/auto-instrumentations-node

You can let OpenTelemetry to instrument your application automatically or do it explicitly by installing required instrumentations, for example, OpenTelemetry Express instrumentation:

# Using npm
npm install @opentelemetry/instrumentation-http @opentelemetry/instrumentation-express

# Using yarn
yarn add @opentelemetry/instrumentation-http @opentelemetry/instrumentation-express

Instrumenting Express.js with OpenTelemetry

After installing OpenTelemetry, you need to configure OpenTelemetry SDK to export data to an OpenTelemetry backend for strorage and visualization:

'use strict'

const otel = require('@opentelemetry/api')
const { BatchSpanProcessor } = require('@opentelemetry/sdk-trace-base')
const { Resource } = require('@opentelemetry/resources')
const { NodeSDK } = require('@opentelemetry/sdk-node')
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-http')
const { AWSXRayIdGenerator } = require('@opentelemetry/id-generator-aws-xray')
const { HttpInstrumentation } = require('@opentelemetry/instrumentation-http')
const { ExpressInstrumentation } = require('@opentelemetry/instrumentation-express')

const dsn = process.env.UPTRACE_DSN
console.log('using dsn:', dsn)

const exporter = new OTLPTraceExporter({
  url: 'http://localhost:14318/v1/traces',
  headers: { 'uptrace-dsn': dsn },
  compression: 'gzip',
})
const bsp = new BatchSpanProcessor(exporter, {
  maxExportBatchSize: 1000,
  maxQueueSize: 1000,
})

const sdk = new NodeSDK({
  spanProcessor: bsp,
  resource: new Resource({
    'service.name': 'myservice',
    'service.version': '1.0.0',
  }),
  idGenerator: new AWSXRayIdGenerator(),
  instrumentations: [new HttpInstrumentation(), new ExpressInstrumentation()],
})
sdk.start()

To avoid potential issues, it is strongly recommended to put the OpenTelemetry initialization code into a separate file called otel.js and use the --require flag to load the tracing code before the application code:

# JavaScript
node --require ./otel.js app.js

# TypeScript
ts-node --require ./otel.ts app.ts

See OpenTelemetry Express.js example for details.

What is Uptrace?

Uptrace is a popular DataDog alternative designed to monitor and analyze the performance of microservices-based architectures.

Uptrace provides end-to-end visibility into the flow of requests and transactions across a distributed system, enabling developers and operators to understand and optimize the behavior of their applications.

You can get started with Uptrace by downloading a DEB/RPM package or a pre-compiled Go binary.

What's next?

By integrating OpenTelemetry with Express.js, you can gain insight into your application's distributed traces, OpenTelemetry metrics, and logs. This helps you understand the behavior of your Express.js application across multiple services or microservices, and facilitates troubleshooting and performance optimization.

Next, instrument more operations to get a more detailed picture. Try to prioritize network calls, disk operations, database queries, error and logs.

You can also create your own instrumentations using OpenTelemetry JS Tracing API.

Blog