Day 13: CI for a Python project of the National Institute of Standards and Technology
Gabor Szabo
Posted on December 13, 2022
Interesting coincidences.
Just a few days ago as I started to work on the Open Source Developer Course I started
to collect Open Source projects by governments and today the first project I found on PyDigger that I wanted to try to contribute to was a project of the NIST - the National Institute of Standards and Technology.
Side note: I find it funny that in this age of globalization when anyone can easily bump into any organization (barring language barrier), organizations still call themselves "National" without including an indication of which nation
, in their name. At least on the web site of NIST you can see US Department of Commerce
in their logo.
The process
Anyway, what did I do?
I noticed that the project has a folder called tests
but that it also has a file called runtests.py
.
I am not sure why do some projects have their own test-runner instead of just running pytest
, but I seem to recall I saw this in other Django-related projects as well.
So I set started a Docker container that I use for better isolation of the foreign code I run from my computer and ran
python runtests.py
Not surprisingly it failed complaining about missing Django.
So I went ahead and installed all the requirements I could find:
pip install -r requirements.txt
pip install -r requirements.core.txt
Running again
python runtests.py
still failed. This time it was missing psycopg2
which is the Python package to connect to PostgreSQL.
So I tried to install it:
pip install psycopg2
It failed, but it pointed me at the installation documentation.
That's a very nice touch!
From that website I understood I need to install libpq
. So I ran
apt-cache search libpq
(I use an Ubuntu 22.10 based Docker container for my experiments.)
In the results of this command I saw libpq-dev
. This is the development package. Libraries usually need that so I installed it:
apt-get install -y libpq-dev
and ran the installation again:
pip install psycopg2
After that I ran the tests again:
python runtests.py
The output looked very much like an output from a test-run and it ended with
Ran 86 tests in 0.271s
It was not hard at all.
I created the GitHub Actions configuration file using a Docker container and configured python 3.11. I let the developers of the package decide which other versions of Python they might want to test with and if they might want to test on Windows and macOS as well.
I sent the Pull-Request
GitHub Actions
name: CI
on:
push:
pull_request:
workflow_dispatch:
schedule:
- cron: '42 5 * * *'
jobs:
test:
strategy:
fail-fast: false
matrix:
python-version: ["3.11"]
runs-on: ubuntu-latest
name: OS Python ${{matrix.python-version}}
container: python:${{matrix.python-version}}
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Install dependencies
run: |
apt-get install -y libpq-dev
pip install -r requirements.txt
pip install -r requirements.core.txt
pip install psycopg2
- name: Check Python version
run: python -V
- name: Run tests
run: python runtests.py
Conclusion
I am already quite experienced setting up CI in general and GitHub Actions in particular, but I think even people who are new to this area could do similar task within a few hours. So I think these will be reasonable assignments in the Open Source Development Courses.
Posted on December 13, 2022
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related
December 13, 2022