After coding Python for quite some time, today I needed to do something that should be pretty straightforward, but it took almost half an hour to figure it out, and the result wasn’t as expected.

Content

• A simple problem
• What about production?
• Light at the end of the tunnel

A simple problem

The problem presented itself when it was necessary to dynamically set a log level to a database connection based on an environment variable. As ORM, the project uses peewee which allows us to leverage Python’s standard logging lib to intercept a peewee log stream and do some manipulation such as changing the log level of the stream.

According to the peewee’s doc, when in need to print all queries to the stderr, we just need to implement the following.

import logging

logger = logging.getLogger('peewee')
logger.addHandler(logging.StreamHandler())
logger.setLevel(logging.DEBUG)

What about production?

Once I read the code I thought “When deploying this to production another log level, such as ERROR, would be more appropriated to avoid showing unnecessary information from the database”.

It’s important to carefully set up your logs to keep the application security risks to a minimum, since a log level with too much information can expose the database’s structure and sensitive data.

Light at the end of the tunnel

The approach chosen to solve this was to add an environment variable to the .env file called DB_LOG_LEVEL, that way the application could vary between DEBUG and ERROR levels of log based on the environment.

Next, we need a way to apply some metaprogramming to dynamically call logging.DEBUG, since the log level the application is taking as the truth comes from env("DB_LOG_LEVEL").

For that, we can use the built-in getattr(…) function, that would return the value of the named attribute of an object, where the name must be a string. With knowledge of this function, let's see how our code would look like now.

import logging
from os import getenv

level = getenv("DB_LOG_LEVEL", "DEBUG")

logger = logging.getLogger("peewee")
logger.addHandler(logging.StreamHandler())
logger.setLevel(getattr(logging, level))

I also added a validation that checks if the value returned by the getenv(...) function matches the log levels present in the logging library, which presents the following implementation.

import logging
from os import getenv

"""
---------------------------------------------------------------------------
 Database logging
--------------------------------------------------------------------------

Database logging will define when and how log info will be shown while
interacting with the database. Accepted log levels are CRITICAL, ERROR,
WARNING, INFO, DEBUG and NOTSET.
"""


LOG_LEVELS = [
    "CRITICAL",
    "FATAL",
    "ERROR",
    "WARNING",
    "WARN",
    "INFO",
    "DEBUG",
    "NOTSET",
]

LOG_LEVEL = getenv("DB_LOG_LEVEL", "DEBUG")

if LOG_LEVEL not in LOG_LEVELS:
    raise Exception(f"DatabaseLogError: Log level '{LOG_LEVEL}' not defined")

logger = logging.getLogger("peewee")
logger.addHandler(logging.StreamHandler())
logger.setLevel(getattr(logging, LOG_LEVEL))
logging.DEBUG

Using metaprogramming to solve this problem, allows us to dynamically access a property of the object logging to pass a valid log level into the logger.setLevel(...) function.

I'm Jack Miras, a Brazilian engineer who works mostly with infrastructure and backend development, for more of my stuff click here.

Happy coding!