In the ever-evolving world of software development, the importance of testing to ensure quality cannot be overstated. This rings especially true when it comes to distributed applications with loosely coupled microservices. These kinds of distributed applications bring a host of complexities that require a thoughtful approach to testing. With the advent of testing tools provided by Skyramp, combined with technology building blocks like Kubernetes and OpenAPI, test generation for distributed apps is easier and more efficient than ever before. This blog will dive into the process of generating tests for your distributed applications by showing a step-by-step example using Skyramp and the aforementioned standards.

Talking About Test Generation

Before we delve into the step-by-step walkthrough of generating tests, let's first understand why test generation is beneficial for distributed applications. Distributed applications, most often composed of various microservices, can be particularly intricate in terms of communication, dependencies, and data flow. Manually writing tests for each component can be time-consuming and prone to human error. Automated test generation can streamline this process, ensuring comprehensive coverage and robust testing.

OpenAPI, formerly known as Swagger, is a set of specifications that defines a standard, language-agnostic interface to RESTful APIs. It's often used to document and describe the capabilities of a microservice, including the available endpoints, input/output parameters, and response formats. This can act as the foundation for automated test generation by providing a clear and structured definition of a service.

Leveraging OpenAPI, Skyramp can analyze your microservices and automatically create test cases based on your API specifications and service dependencies. This allows you to verify that the various components of your distributed application work seamlessly together, saving you time. It should be noted that Skyramp supports other protocols for test generation like gRPC and Protocol Buffers, but we'll save that for another day.

Generating Tests: A Step-By-Step Walkthrough

Now, let's get into the nitty-gritty of how to generate tests for your distributed applications using Skyramp. We will use our tried and true sample-microservices repo on GitHub, which is an 11-tier distributed app with REST support already configured for deployment to a Kubernetes cluster. The app itself is a complete web-based e-commerce store created by Google, known as Online Boutique.

Below is the step-by-step walkthrough to guide you along the journey. These steps were tested on Mac, but should work on Linux or Windows with a bash shell.

Step 1: Install the Skyramp CLI

The first step is to install the Skyramp Command Line Interface (CLI). See the Skyramp Docs for specifics, or you can run the following command in your terminal:

bash -c "$(curl -fsSL https://skyramp.dev/installer.sh)"

NOTE: You will also need Docker installed to continue.

Step 2: Clone the Repository

Next, clone the sample repository discussed above. This contains the code and configuration files necessary for your distributed application.

git clone https://github.com/letsramp/sample-microservices.git

Step 3: Create a Cluster

Now, create a cluster using the Skyramp CLI. This cluster will serve as the environment in which your microservices will be deployed and tested.

skyramp cluster create -l

Step 4: Change to the Demo Directory

Navigate to the directory containing the prepared REST demo project under "rest-demo".

cd sample-microservices/skyramp/rest-demo

Step 5: Deploy the Cluster with Skyramp Deployer

Use the Skyramp Deployer to bring up the microservices within the cluster.

skyramp deployer up rest-app

The output will look like this:

ad-service-7cb787976d-l49n5                    ready
currency-service-5b966cbfb8-q7dvf              ready
email-service-745ddcd9c4-7wdvq                 ready
cart-service-56cd75c447-xzdlk                  ready
checkout-service-c5546db56-lsdtm               ready
product-catalog-service-5467566c8f-sl6wb       ready
skyramp-worker-6c8bfdf587-jr2tq                ready
payment-service-5f9b47cfd4-cw4j2               ready
recommendation-service-6f8cd558b8-dmvh8        ready
shipping-service-599947db74-8dtkn              ready
frontend-6f8fc574f7-nv76g                      ready
redis-6fc54f68c4-v6mqb                         ready
All pods are ready.

Step 6: OpenAPI Configuration

In this project, there is a pre-defined OpenAPI specification file (demo.yaml). This file describes the API endpoints and their expected behavior. Take a peek at the contents of this file.

cat openapi/demo.yaml

Notice that the definition centers around charging a credit card. We can use this definition to test the payment service running in our cluster.

Step 7: Generate a Test

Now comes the fun part - generating a test using the Skyramp CLI. You can specify the protocol, API schema, alias, and port for your test. Run the command below to see how this works.

skyramp tester generate \
--protocol openapi \
--api-schema openapi/demo.yaml \
--alias test-rest-demo \
--port 60000

This command will automatically generate test cases for your distributed application based on the provided OpenAPI schema.

Step 8: Tune Your Test

Upon running the test generation command from the step above, Skyramp will create three files that contain your test cases and configuration. You can review and customize these files to fit your specific testing needs. The output will look something like this:

Creating test files     [##############################################################] 100 %
Successfully created/updated the test description configuration files:
    endpoints/test-rest-demo-tDPT.yaml
    scenarios/scenario-test-rest-demo-tDPT.yaml
    tests/test-test-rest-demo-tDPT.yaml

Let's make a few edits to the files that were generated.

1) Change the serviceName value in the generated "endpoint" file to use the payment service running in the cluster:

serviceName: payment-service

2) Change the requestName parameter in the generated "test" file to scenarioName and assign the value "scenario_123".

 - scenarioName: scenario_123

3) Modify the generated "scenario" file to include a scenarios section at the end of the file that looks like this:

scenarios:
  - name: scenario_123
    steps:
      - requestName: chargeRequest
      - asserts: requests.chargeRequest.res.transaction_id != null

4) Also modify generated parameters and values in the "scenario" file under the requests section as shown below. Many of the parameters will be auto-generated based on the OpenAPI definition, but we can improve the values.

    - name: chargeRequest
      headers:
        Content-Type: "application/json"
      blob: |-
        {
          "amount": {
            "currency_code": "USD",
            "units": 100
          },
          "credit_card": {
            "credit_card_cvv": 100,
            "credit_card_expiration_month": 10,
            "credit_card_expiration_year": 2030,
            "credit_card_number": "4432-8015-6152-0454"
          }
        }

Step 9: Run and Refine

After completing the edits from above, we are ready to run our test against the payment service. Simply run Skyramp Tester to execute the test scenario we composed.

skyramp tester start test-test-rest-demo-2VHL -n test-rest-demo

The output should indicate a passed test such as this:

Starting tests
Tester finished
Test test-rest-demo------
 [Status: finished] [Started at: 2023-10-26 09:32:56 PDT] [End: 2023-10-26 09:32:57 PDT] [Duration: 1s]
  - pattern0.scenario_123
    [Status: finished] [Started at: 2023-10-26 09:32:57 PDT] [Duration: 0s]
  - pattern0.scenario_123.0.chargeRequest
    [Status: finished] [Started at: 2023-10-26 09:32:57 PDT] [Duration: 0s]
    Executed: {"transaction_id":"e740f1b8-3618-4253-94ae-9be0de082f68"}
  - pattern0.scenario_123.1.assert
    [Status: finished] [Started at: N/A]
    Assert: requests.chargeRequest.res.transaction_id != null
    Passed: true

You can continue to fine-tune your tests by modifying the generated files. Working with these files, you can see the modular approach Skyramp takes to testing. Adjust the test cases, add custom scenarios, and configure test environments as needed to ensure comprehensive testing for your distributed applications.

This Was Only a Test

In conclusion, automated test generation with Skyramp simplifies the testing process for distributed applications. It enhances the efficiency and reliability of testing scenarios, ensuring that your microservices work harmoniously to deliver a robust and error-free application.

By following the step-by-step walkthrough outlined in this blog, you can see how to effectively generate and customize tests for your distributed apps, saving time and ensuring the quality of your software. Happy testing with Skyramp!