When using Heroku CI for automatic testing, the default output is just pass or fail without any parsing of how many tests passed or what errors where found.

This can be improved if you output the results in the Test Anything Protocol-format (TAP-format) according to Heroku.

This format is not built into the test frameworks used in Ruby on Rails. Therefore I had to write my own test reporter.

I chose to implement this for Minitest and decided to use the gem
minitest-reporters.

Expected outcome

I have a small project where the test suit looks like this:

To adhere to the TAP-format I need it to also output:

TAP version 13
1..7
ok 1 - AccountsControllerTest - test_renders_users_account
ok 2 - AccountsControllerTest - test_require_login
ok 3 - CallbacksMessagesControllerTest - test_handle_status_callback
ok 4 - MessagesControllerTest - test_should_create_message
ok 5 - ContactsControllerTest - test_should_create_contact
ok 6 - ContactsControllerTest - test_should_update_contact
ok 7 - ContactsControllerTest - test_show_contact

Solution

I created a custom reporter called TapReporter by inspecting a standard reporter from the minitest-reporters gem called ProgressReporter (see source):

The solution on Heroku

A failing test

Successful tests

Keywords

• Ruby on Rails.
• Heroku - Platform as a Service, useful for hosting web applications.
• Continuous integration (CI) - in this post it mostly refers to automated testing.
• Test anything protocol(TAP) - a protocol for test results.
• Minitest - a test framework for ruby.