Docker opens up access to an incredible ecosystem of build-up applications. We find containers for everything … We can run a postgres database, run a redis database or launch an ftp server.

Docker is a great way to run dependency our code needs when running system integration tests that will validate our application.

In this article, we will discover how to use docker in our python tests by writing a minimum of boilerplate while keeping the same CI/CD pipeline.

Testing an application beyond its code thanks to docker

A bug can hide in the use of a database, for example, a badly written SQL query. We need a database to test that. We can’t validate this behavior with just the code.

It is an important topic. Django, a major python framework, provides a partial answer to this. It only manages a test workflow for the database. This workflow also requires you to launch the database by yourself before launching your tests.

To solve this topic, our tests must be able to launch containers by themselves.

To mount the containers from our tests, we will boost our tests with Fixtup. Our tests will be able to start and stop containers at the right time by themselves.

If a container is expensive to start, it can be mounted once and reused. pytest will stop it after playing the last test it has to play.

Install the requirements

To follow this article, we need to prepare our developper station.

1 . We need to be able to use docker and docker-compose from a terminal.

$ docker --version
$ docker-compose --version

2 . We need poetry to create the python project

$ poetry

Create a python project

$ poetry init
$ poetry add psycopg2-binary
$ poetry add --dev pytest fixtup

We will create a docker container for postgresql as a fixture to use in our tests.

$ poetry run fixtup init
Choose a directory to store fixture templates : tests/fixtures
Python manifest (pyproject.toml) [pyproject.toml]

$ poetry run fixtup new
Choose a fixture identifier : postgres
Mount environment variables on this fixture (y/n) [n]
Mount docker container on this fixture (y/n) [n] y

tests/fixtures/postgres/docker-compose.yml

version: "3.9"
services:
    postgresql:
        init: true
        image: postgres
        ports:
            - "5432:5432"
        environment:
            - POSTGRES_PASSWORD=1234
        volumes:
            - /var/lib/postgresql

We will activate the hook_started hook to wait for the database server to be loaded before starting our test.

tests/fixtures/postgres/.hooks/hook_started.py

from time import sleep
import fixtup. helper

fixtup.helper.wait_port(5432, timeout=2000)

# we wait 1 second to ensure the database is ready
sleep(1)

Using postgresql database in a test

Now we are going to install the driver for postgresql and write our first test.

test/test_postgresql_database.py

import fixtup
import psycopg2

def test_postgres_should_work():
    with fixtup.up('postgres'):
        conn = psycopg2.connect("host=127.0.0.1 dbname=postgres user=postgres password=1234")
        core = conn.cursor()
        cur.execute('SELECT 1+1')
        res = cur.fetchall()
        assert res[0][0] == 2
        conn.close()

Use the same postgresql database for all tests

Containers can take a long time to start, that’s why we would like to reuse them between several tests. By default, fixtup stops and starts them between each test. By activating the keep_up policy of fixtup, the containers will remain up as soon as they have been launched. They will be deleted when pytest has finished running.

tests/fixtures/postgres/fixtup.yml

# This flag control if a fixture stay up and running between every test. The fixture is stop and
# unmount when the test process stop
#
# This attribute allow to start a database only once and stop the container only when unittest has finished to run
# the test suite. It may be interested to improve the performance if your start and stop process is too slow
keep_up: true

Wait for database

Instead of waiting for port availability and a timer after to wait database to be ready, we will rewrite the hook_started to attempt to connect to the database regularly.

tests/fixtures/postgres/.hooks/hooks_started.py

from time import monotonic
import psycopg2

start = monotonic()
connected = False
timeout = 5
while not connected:
    try:
        conn = psycopg2.connect("host=127.0.0.1 dbname=postgres user=postgres password=1234")
        connected = True
        conn.close()
    except Exception:
        if monotonic() - start > timeout:
            raise TimeoutError()

Going further

In this article, we have seen how to use docker containers in our tests with pytest and fixtup.

If your test depends on an application that works in docker, pytest can mount these containers through Fixtup. It can be a redis base, a localstack stack for AWS, or an ftp server...

If you're stepping through a test from pycharm or vscode, you can debug your test and also the fixture hooks.

We haven't seen how to mount the database schema at startup, nor how to clean up the database between each test. Fixtup hooks allow you to do this.

I plan to develop a fixtup plugin for sqlalchymie and another for django that will allow you to use a database without writing a specific hook.