Structured logging best practices

In structured logging, log messages are broken down into key-value pairs, making it easier to search, filter, and analyze logs. This is in contrast to traditional logging, which usually consists of unstructured text that is difficult to parse and analyze.

What is structured logging?

Structured logging is the practice of capturing and storing log messages in a structured and organized format.

Traditional logging often involves printing raw text messages to log files, which can be difficult to parse and analyze programmatically.

In contrast, structured logging formats log messages as key-value pairs or in structured data formats such as JSON or XML.

Why to use structured logging?

Structured logging provides the following benefits:

Improved readability. The structured format makes log messages more human-readable, allowing developers and operators to easily understand the content without relying solely on raw text parsing.
Better searching and filtering. Structured data makes it easier to search for specific log entries or filter logs based on specific criteria. This is especially useful for large-scale applications with large amounts of log data.
Easy integration with tools. Structured logs can be ingested and processed by various log management and analysis tools, enabling powerful analysis, visualization, and monitoring of application behavior.
Improved debugging and troubleshooting. When log messages are structured, it is easier to include relevant contextual information, such as timestamps, error codes, and specific attributes related to the logged events, which facilitates effective debugging and troubleshooting.
Consistency and scalability. Structured logging promotes a consistent and uniform log format throughout the application, making it easier to scale logging capabilities and maintain logs in a standardized manner.

Structured log formats

Structured logging can be implemented using various data formats, with JSON being one of the most commonly used due to its simplicity and human-readability.

However, other formats can also be used depending on the requirements of the application and the logging framework in use.

JSON format

JSON is a lightweight data exchange format that is easy for both humans and machines to read and write. It represents data as key-value pairs and arrays, making it an excellent choice for structured logging due to its simplicity and widespread support.

Example:

{
  "timestamp": "2023-08-04T12:34:56Z",
  "level": "INFO",
  "module": "payment-service",
  "message": "Payment successful",
  "amount": 50.25,
  "transaction_id": "abc123"
}

You can also use JSON to include structured data in your log messages:

request failed {"http.method": "GET", "http.route": "/users/:id", "enduser.id": 123, "foo": "hello world"}

logfmt

This format represents log entries as a series of key-value pairs separated by delimiters such as spaces or tabs. It is simple and easy to implement.

If a value contains a space, you must enclose it in quotation marks. For example:

request failed http.method=GET http.route=/users/:id enduser.id=123 foo="hello world"

Free format

If your library does not support structured logging, you can still improve grouping by quoting params:

# good
can't parse string: "the original string"
"foo" param can't bempty

# bad
can't parse string: the original string
foo param can't be empty

Logging backend

Uptrace is an open source APM for OpenTelemetry that supports logs, traces, and metrics. You can use it to monitor applications and troubleshoot issues.

Uptrace natively supports structured logging and automatically parses log messages to extract the structured data and store it as attributes.

Uptrace comes with an intuitive query builder, rich dashboards, alerting rules, notifications, and integrations for most languages and frameworks.

Uptrace can process billions of logs on a single server and allows you to monitor your applications at 10x lower cost.

In just a few minutes, you can try Uptrace by visiting the cloud demo (no login required) or running it locally with Docker. The source code is available on GitHub.

Conclusion

Structured logging enables better log management, improved troubleshooting, and better application monitoring, resulting in more efficient and reliable software development and maintenance processes.

Blog