Play with Testinfra which tests states of infrastructure

This is my note about Testinfra which can be an alternative of ServerSpec.
I previously wrote about goss which is also a testing tool for infrastructure.
This might be nice to compare with.

Work with Serverspec alternative tool Goss

koh-sh ・ Nov 8 '19 ・ 4 min read

#go #testing #linux

What is Testinfra

https://testinfra.readthedocs.io/en/latest/

About
With Testinfra you can write unit tests in Python to test actual state of your servers configured by management tools like Salt, Ansible, Puppet, Chef and so on.

Testinfra aims to be a Serverspec equivalent in python and is written as a plugin to the powerful Pytest test engine

To put it simply, Testinfra is like ServerSpec but written in Python, and works great with configuration management tools like Ansible, Chef.

Features

Compared with ServerSpec, Testinfra has the below features.

Lots of Connection Backend

Testinfra has several options for connecting remote hosts other than plain SSH.
Testinfra can refer to an inventory of Ansible, also able to connect containers by docker or kubectl.

https://testinfra.readthedocs.io/en/latest/backends.html

local
paramiko
docker
ssh
salt
ansible
kubectl
winrm
LXC/LXD

Above is the list of connection backend.

Interactive testing

https://testinfra.readthedocs.io/en/latest/api.html

It can connect to host and test interactively so you don't even have to write a test file.
And I think Testinfra works great with ipython.

[koh@kohs-MBP] ~/vag_test
% ipython
Python 3.7.3 (default, May  1 2019, 16:07:48)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.5.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import testinfra

In [2]: host = testinfra.get_host("paramiko://vagrant@Vag1:22", sudo=True)

In [3]: host.file("/etc/passwd").mode == 0o644
Out[3]: True

In [4]:

Let's try

Environment

Client

MacOS Mojave 10.14.5
Python 3.7.3

Remote host

CentOS 7.5.1804 on Vagrant
Python 2.7.5

Install

Installing with pip on the client machine.

$ pip install testinfra

writing a test

Writing a test file.

def test_passwd_file(host):
    passwd = host.file("/etc/passwd")
    assert passwd.contains("root")
    assert passwd.user == "root"
    assert passwd.group == "root"
    assert passwd.mode == 0o644


def test_nginx_running_and_enabled(host):
    nginx = host.service("nginx")
    assert nginx.is_running
    assert nginx.is_enabled


def test_selinux(host):
    cmd = host.run("/usr/sbin/getenforce")
    assert cmd.stdout == "Enforcing\n"

In this test example, it tests

contents, owner/permission of /etc/passwd
is-active, is-enabled of NGINX
status of selinux

Test execution

So from here, I am running Testinfra.
Testing from local machine to CentOS on Vagrant with SSH.
Also, NGINX is running on the remote machine.

[koh@kohs-MBP] ~/vag_test
% py.test -v test_first.py --connection=ssh --host=Vag1
======================================== test session starts ========================================
platform darwin -- Python 3.7.3, pytest-4.4.1, py-1.8.0, pluggy-0.9.0 -- /Users/koh/.pyenv/versions/3.7.3/bin/python3.7
cachedir: .pytest_cache
rootdir: /Users/koh/vag_test
plugins: xonsh-0.8.12, testinfra-3.0.5
collected 3 items

test_first.py::test_passwd_file[ssh://Vag1] PASSED                                            [ 33%]
test_first.py::test_nginx_running_and_enabled[ssh://Vag1] PASSED                              [ 66%]
test_first.py::test_selinux[ssh://Vag1] PASSED                                                [100%]

===================================== 3 passed in 3.22 seconds ======================================
[koh@kohs-MBP] ~/vag_test
%

It is clear and easy to check the result.
And now I stop NGINX.

[koh@kohs-MBP] ~/vag_test
% py.test -v test_first.py --connection=ssh --host=Vag1
======================================== test session starts ========================================
platform darwin -- Python 3.7.3, pytest-4.4.1, py-1.8.0, pluggy-0.9.0 -- /Users/koh/.pyenv/versions/3.7.3/bin/python3.7
cachedir: .pytest_cache
rootdir: /Users/koh/vag_test
plugins: xonsh-0.8.12, testinfra-3.0.5
collected 3 items

test_first.py::test_passwd_file[ssh://Vag1] PASSED                                            [ 33%]
test_first.py::test_nginx_running_and_enabled[ssh://Vag1] FAILED                              [ 66%]
test_first.py::test_selinux[ssh://Vag1] PASSED                                                [100%]

============================================= FAILURES ==============================================
____________________________ test_nginx_running_and_enabled[ssh://Vag1] _____________________________

host = <testinfra.host.Host object at 0x103826438>

    def test_nginx_running_and_enabled(host):
        nginx = host.service("nginx")
>       assert nginx.is_running
E       assert False
E        +  where False = <service nginx>.is_running

test_first.py:11: AssertionError
================================ 1 failed, 2 passed in 3.06 seconds =================================
zsh: exit 1     py.test -v test_first.py --connection=ssh --host=Vag1
[koh@kohs-MBP] ~/vag_test
%

It is obvious that which item is failed and how it failed.
Also able to test multiple hosts.

[koh@kohs-MBP] ~/vag_test
% py.test -v test_first.py --connection=ssh --host=Vag1,Vag2
======================================== test session starts ========================================
platform darwin -- Python 3.7.3, pytest-4.4.1, py-1.8.0, pluggy-0.9.0 -- /Users/koh/.pyenv/versions/3.7.3/bin/python3.7
cachedir: .pytest_cache
rootdir: /Users/koh/vag_test
plugins: xonsh-0.8.12, testinfra-3.0.5
collected 6 items

test_first.py::test_passwd_file[ssh://Vag1] PASSED                                            [ 16%]
test_first.py::test_nginx_running_and_enabled[ssh://Vag1] PASSED                              [ 33%]
test_first.py::test_selinux[ssh://Vag1] PASSED                                                [ 50%]
test_first.py::test_passwd_file[ssh://Vag2] PASSED                                            [ 66%]
test_first.py::test_nginx_running_and_enabled[ssh://Vag2] FAILED                              [ 83%]
test_first.py::test_selinux[ssh://Vag2] FAILED                                                [100%]

============================================= FAILURES ==============================================
____________________________ test_nginx_running_and_enabled[ssh://Vag2] _____________________________

host = <testinfra.host.Host object at 0x11026f4e0>

    def test_nginx_running_and_enabled(host):
        nginx = host.service("nginx")
>       assert nginx.is_running
E       assert False
E        +  where False = <service nginx>.is_running

test_first.py:11: AssertionError
_____________________________________ test_selinux[ssh://Vag2] ______________________________________

host = <testinfra.host.Host object at 0x11026f4e0>

    def test_selinux(host):
        cmd = host.run("/usr/sbin/getenforce")
>       assert cmd.stdout == "Enforcing\n"
E       AssertionError: assert 'Permissive\n' == 'Enforcing\n'
E         - Permissive
E         + Enforcing

test_first.py:17: AssertionError
================================ 2 failed, 4 passed in 6.14 seconds =================================
zsh: exit 1     py.test -v test_first.py --connection=ssh --host=Vag1,Vag2
[koh@kohs-MBP] ~/vag_test
%

Conclusion

Compared with ServerSpec, ServerSpec has more resources, and many more documents on the internet (especially in Japanese).
But as I mentioned there are some pros of Testinfra so if you prefer Python, it is worth trying at least once.

Blog