The Cover-Agent test coverage tool just dropped!!

So, I'll be walking you through everything you need to know in order to get Cover-Agent up and running locally on your workstation.

Overview of Cover-Agent

First, let's understand what the Cover-Agent is and what it does.

The Cover-Agent is an opensource test coverage tool by CodiumAI that utilizes various Generative AI models to automate unit test creation for your projects.
Under the hood, the Cover-Agent is comprised of various components, namely;

• The Test Runner: Which runs the test suite and generates code coverage reports.
• The Coverage Parser: Which verifies that code coverage increases as tests are added, ensuring that new tests contribute to the overall test effectiveness.
• The Prompt Builder: Which gathers the necessary data from the codebase and constructs the prompt to be passed to the Large Language Model.
• The Caller: Which interacts with the LLM to generate tests based on the prompt provided.

Cover-Agent can currently be run locally on your terminal, although CodiumAI is working on making it compatible with popular CI platforms.

Installation

Prerequisites
Before you begin, ensure you have the following:

• OPENAI_API_KEY: Required for calling the OpenAI API.
• Python/PIP: Required for installing Cover-Agent.

To get Cover-Agent running as fast as possible, the easiest setup would be to install Cover-Agent as a Python Pip package using the command.

 bash
$ pip install git+https://github.com/Codium-ai/cover-agent.git

To ensure that Cover-Agent have been properly installed, you can run the cover-agent command on the terminal, and you should see something like this.

 bash
$ cover-agent
usage: cover-agent [-h] --source-file-path SOURCE_FILE_PATH --test-file-path TEST_FILE_PATH --code-coverage-report-path CODE_COVERAGE_REPORT_PATH --test-command TEST_COMMAND
                   [--test-command-dir TEST_COMMAND_DIR] [--included-files [INCLUDED_FILES ...]] [--coverage-type COVERAGE_TYPE] [--report-filepath REPORT_FILEPATH]
                   [--desired-coverage DESIRED_COVERAGE] [--max-iterations MAX_ITERATIONS] [--additional-instructions ADDITIONAL_INSTRUCTIONS] [--openai-model OPENAI_MODEL] [--prompt-only]
cover-agent: error: the following arguments are required: --source-file-path, --test-file-path, --code-coverage-report-path, --test-command

Next, you need to set your OpenAI key in your environment variables.

 bash
$ export OPENAI_API_KEY=**********************

Generating Tests using Cover-Agent

Now you are ready to run the Cover Agent to generate and validate unit tests.
You can execute it from the command line using the following command:

 bash
$ cover-agent \
  --source-file-path "<path_to_source_file>" \
  --test-file-path "<path_to_test_file>" \
  --code-coverage-report-path "<path_to_coverage_report>" \
  --test-command "<test_command_to_run>" \
  --test-command-dir "<directory_to_run_test_command>" \
  --coverage-type "<type_of_coverage_report>" \
  --desired-coverage <desired_coverage_between_0_and_100> \
  --max-iterations <max_number_of_llm_iterations> \
  --included-files "<optional_list_of_files_to_include>"

Running Cover-Agent

I have a Python project which auto-generates portfolios using the GitHub API. The API endpoints in this project are located in my app.py file.

I also have a test file test_app.py, for testing the API endpoints.

So with this setup I’m ready to run cover-agent from the terminal.

I ran the command below to generate relevant unit tests for my project.

 bash
$ cover-agent \
  --source-file-path "app.py" \
  --test-file-path "test_app.py" \
  --code-coverage-report-path "coverage.xml" \
  --test-command "pytest --cov=. --cov-report=xml --cov-report=term" \
  --test-command-dir "." \
  --coverage-type "cobertura" \
  --desired-coverage 70 \
  --max-iterations 10

Ensure you replace the file paths with the actual file path where your code and test files are located.

Changes

Cover-Agent appends the generated unit tests to your test file.

After generating tests for your project, you’ll notice some new files in your repo.

• generated_prompt.md: Which contains the full prompt that is sent to the LLM.

• coverage.xml: Which contains the test coverage scope.

• run.log: Which contains a copy of the logger that gets dumped to your stdout.

• test_results.html: Which contains the results table that contains the following for each generated test:

• Test status
• Failure reason (if applicable)
• Exit code,
• stderr
• stdout
• Generated test

Testing is no doubt one of the most important aspects of software development. And with a few steps you can up your testing game by utilizing the Cover-Agent tool to generate meaningful tests for your projects, ensuring test coverage for your codebase in no time.

Cover-Agent is an opensource software on GitHub, which means you can take a look at the inner workings of the tool, make meaningful contributions and help extend the capabilities of the tool.

References

• Cover-Agent GitHub Repo
• Cover-Agent Releases