Integration Testing for GraphQL APIs, type-safe, run locally and in CI

We've recently built a testing framework for GraphQL APIs.
It's open source and available as part of our monorepo.
I'd like to take the chance to explain the motivation behind it, how it works, and how you can use it to test your own GraphQL APIs locally and within your CI.

Actually, it's not just good for testing GraphQL APIs,
as you can also use it to test REST APIs, Databases and more,
but I'll focus on GraphQL APIs in this article.

Motivation

WunderGraph is a crossover between a Backend for Frontend (BFF) and a (GraphQL) API Gateway.
We use GraphQL as the main interface to APIs and Databases.
You can introspect one or more services into what we call the "Virtual Graph".
The Virtual Graph is a composed GraphQL Schema across all your services.

By defining a GraphQL Operation, you're essentially creating a "Materialized View" on top of the Virtual Graph.
Each of these views, we call them simply "Operations", is exposed as a JSON RPC Endpoint.
That's why we call the graph a "Virtual Graph", because we're not really exposing it.

If you're building such a system, you'll want to make sure that it integrates well with all sorts of services.
Especially when you allow your users to create Materialized Views across multiple services or data sources,
you want to make sure that all of these integrations work as expected and that you don't break them with future changes.

How it works: Type-safe testing for GraphQL APIs

You can probably roll your own testing framework for GraphQL APIs,
but WunderGraph comes with some unique features that make it really easy to write tests for GraphQL APIs.

Let's start with a simple GraphQL API.

Testing a simple monolithic GraphQL API

First, we need to tell WunderGraph about our GraphQL API.
We do so by "introspecting" it into the Virtual Graph.

// .wundergraph/wundergraph.config.ts
const countries = introspect.graphql({
    apiNamespace: 'countries',
    url: 'https://countries.trevorblades.com/',
});

// configureWunderGraph emits the configuration
configureWunderGraphApplication({
    apis: [countries],
});

This will add the Countries GraphQL API to our Virtual Graph.
Next, let's define a GraphQL Operation that we want to test.

# .wundergraph/operations/Continents.graphql
query Continents {
    countries_continents {
        name
        code
    }
}

This creates a Continents Operation, but not only that.
We're also generating TypeScript models for this Operation and a type-safe client for it,
which will be quite handy in the next step, when we want to write our tests.

// .wundergraph/test/index.test.ts
import { afterAll, beforeAll, describe, expect, test } from '@jest/globals';
import fetch from 'node-fetch';
import { createTestServer } from '../.wundergraph/generated/testing';

const wg = createTestServer({ fetch: fetch as any });
beforeAll(() => wg.start());
afterAll(() => wg.stop());

describe('test Countries API', () => {
    test('continents', async () => {
        const result = await wg.client().query({
            operationName: 'Continents',
        });
        expect(result.data?.countries_continents.length).toBe(7);
    });
});

That's it.
Before each run, we create a dedicated instance of WunderGraph,
which is responsible to run the GraphQL Engine for the Virtual Graph and the JSON RPC Server.
We then use the generated client to execute the Continents Operation and make sure that we get the expected result.

There are a few details to note here.

You might have noticed that the createTestServer function comes from the .wundergraph/generated/testing module,
so this is actually a generated file.
When we call wg.client(), the returned client is also generated,
meaning that we can easily write type-safe assertions against the result.

This has a few advantages. First, we can use the TypeScript compiler to make sure that we're not making any mistakes.
Second, we can use the TypeScript compiler to guard against breaking changes in the GraphQL API.
When you change the GraphQL API, WunderGraph will re-generate the client and assertions might immediately fail if they are affected by the change.

Another thing to note is that we've designed the testing framework in a way that it's compatible not only with Jest,
but also with other testing frameworks like Mocha, Jasmine, and more.
We've also made sure that tests can run in parallel,
otherwise testing might become a bottleneck in your CI.

To run the tests, we simply run npm test in our case.
The full example can be found here.

If you go through the examples, you'll notice that there are a lot more examples with different setups.
Let me go through some of them to illustrate the different use cases.

Integration Tests for Apollo Federation / Federated GraphQL APIs

WunderGraph supports Apollo Federation, so you can not just use WG as a BFF or API Gateway on top of your federated Subgraphs,
but also use the testing framework to test your federated GraphQL APIs.

Here's an example configuration to introspect a few Subgraphs:

// .wundergraph/wundergraph.config.ts
const federatedApi = introspect.federation({
    apiNamespace: 'federated',
    upstreams: [
        {
            url: 'http://localhost:4001/graphql',
        },
        {
            url: 'http://localhost:4002/graphql',
        },
        {
            url: 'http://localhost:4003/graphql',
        },
        {
            url: 'http://localhost:4004/graphql',
        },
    ],
});

// configureWunderGraph emits the configuration
configureWunderGraphApplication({
    apis: [federatedApi],
});

Next, let's define a GraphQL Operation that we want to test.

# .wundergraph/operations/TopProducts.graphql
query {
    topProducts: federated_topProducts(random: true) {
        upc
        name
        price
        reviews {
            id
            body
            author {
                id
                name
                username
            }
        }
    }
}

And here's the test (abbreviated):

describe('Test federation API', () => {
    test('top products', async () => {
        const result = await wg.client().query({
            operationName: 'TopProducts',
        });
        expect(result.data?.topProducts?.length).toBe(3);
    });
});

In this case, we're validating that we're getting back a list of 3 products.

Testing the Hasura GraphQL Engine

Testing the Hasura GraphQL Engine is a tiny bit more complicated to set up,
because we need to inject a Header for authentication into each request,
which we don't want to commit to our repository.

That said, testing your Hasura GraphQL API is very important.
As Hasura generates a GraphQL API from the database structure,
changing the database structure might break API consumers at any time.
Writing tests against your database-generates GraphQL API will help you to catch these issues early.

This post is about testing GraphQL APIs,
but I'd like to note that one of the great things about WunderGraph is that you can use it to decouple clients from an API implementation with its JSON RPC Middleware layer.

Back to topic, let's configure WunderGraph to introspect the Hasura GraphQL Engine.

// .wundergraph/wundergraph.config.ts
const hasura = introspect.graphql({
    apiNamespace: 'hasura',
    url: 'https://hasura.io/learn/graphql',
    headers: (builder) => builder.addStaticHeader('Authorization', new EnvironmentVariable('HASURA_AUTHORIZATION')),
});
// configureWunderGraph emits the configuration
configureWunderGraphApplication({
    apis: [hasura],
});

Notice that we're using the headers builder to inject a Header into each request from the process environment.
This is a great way to inject secrets into your tests without committing them to your repository.
As WunderGraph generates a configuration file (json) for our Virtual Graph,
we might be leaking secrets if we commit this file to our repository.
By using the EnvironmentVariable builder, we create a placeholder for the secret in the generated config file and inject the secret at runtime.

Next, let's define an operation that we want to test.

# .wundergraph/operations/hasura/Todos.graphql
query ($limit: Int = 3) {
    hasura_todos(limit: $limit) {
        id
        title
        is_public
        is_completed
    }
}

And here's the test (abbreviated):

describe('Test Hasura API', () => {
    test('todos', async () => {
        const result = await wg.client().query({
            operationName: 'hasura/Todos',
        });
        expect(result.data?.hasura_todos?.length).toBe(3);
    });
});

Testing Joins across multiple GraphQL APIs

WunderGraph is capable of joining data across multiple APIs,
so let's test this feature as well.

In this case, we need to configure WunderGraph to introspect two APIs.

// .wundergraph/wundergraph.config.ts
const weather = introspect.graphql({
    apiNamespace: 'weather',
    url: 'https://weather-api.wundergraph.com/',
});

const countries = introspect.graphql({
    apiNamespace: 'countries',
    url: 'https://countries.trevorblades.com/',
});

// configureWunderGraph emits the configuration
configureWunderGraphApplication({
    apis: [weather, countries],
});

Next, we define an operation that fetches a country by its code and joins the weather data for the capital city.

# .wundergraph/operations/CountryWeather.graphql
query ($countryCode: String!, $capital: String! @internal) {
    country: countries_countries(filter: { code: { eq: $countryCode } }) {
        code
        name
        capital @export(as: "capital")
        weather: _join @transform(get: "weather_getCityByName.weather") {
            weather_getCityByName(name: $capital) {
                weather {
                    temperature {
                        max
                    }
                    summary {
                        title
                        description
                    }
                }
            }
        }
    }
}

Finally, we write a test that validates the result.

describe('Test joins', () => {
    test('country weather', async () => {
        const result = await wg.client().query({
            operationName: 'CountryWeather',
            variables: {
                countryCode: 'DE',
            },
        });
        expect(result.data?.country[0].weather.temperature.max).smallerThan(40);
    });
});

I really hope this test will never fail, although it's not unlikely that it will.

Testing Authentication for GraphQL APIs

Another feature that WunderGraph brings to the table is the ability to add authentication to an existing GraphQL API.

Here's an example configuration that adds token-based authentication to any GraphQL API.

// .wundergraph/wundergraph.config.ts
const countries = introspect.graphql({
    apiNamespace: 'countries',
    url: 'https://countries.trevorblades.com/',
});

// configureWunderGraph emits the configuration
configureWunderGraphApplication({
    apis: [countries],
    server,
    operations,
    authentication: {
        tokenBased: {
            providers: [
                {
                    jwksJSON: new EnvironmentVariable('JWKS_JSON'),
                },
            ],
        },
        customClaims: {
            tenantID: {
                jsonPath: 'teid',
            },
        },
    },
});

In this case, we're using an EnvironmentVariable to load a JSON Web Key Set (JWKS) from the process environment.
If you're using a 3rd party authentication provider, JWKS is a great way to share information with a policy enforcement point (PEP) like WunderGraph.

But that's not all. WunderGraph also supports custom claims.
Let's say that you want to use the tenant ID from your JWT to scope your GraphQL API.
You can do this by adding a custom claim to your auth config, and then use the @fromClaim directive to inject the claim into your GraphQL API.
Let's see how this works in a testable way.

# .wundergraph/operations/TenantInfo.graphql
query ($tenantID: String! @fromClaim(name: tenantID)) {
    tenantInfo(id: $tenantID) {
        id
        name
        users {
            id
            name
        }
    }
}

WunderGraph ensures that the resulting JSON RPC API is only accessible to users that have a valid JWT with a tenant ID claim.
A middleware will also enforce that the user doesn't send a tenantID variable as part of the request.

Great, we've built an API that we assume is secure.
Let's ensure that this is actually correct (abbreviated version).

// index.test.ts
describe('Test authentication', () => {
    test('tenant info unauthorized', async () => {
        const result = await wg.client().query({
            operationName: 'TenantInfo',
        });
        expectUnauthorized(result);
    });
    test('tenant info authorized', async () => {
        const client = wg.client();
        client.setAuthorizationToken(tokenWithTenantID('123'));
        const result = await client.query({
            operationName: 'TenantInfo',
        });
        expect(result.data?.tenantInfo.id).toEqual('123');
    });
});

I left out some of the complexity of the test setup,
but I hope you get the idea.

A lot of frameworks give you very little tooling to test the correctness and security of your APIs automatically.
We try to make this as easy as possible with WunderGraph.

Testing your GraphQL APIs in Continuous Integration

Adding to the previous section, we can also use WunderGraph to test our GraphQL APIs in a CI environment.
E.g. you could use GitHub Actions to run your tests on every commit, on a schedule or with a webhook.
Wouldn't it be great if you could automatically test the correctness of your GraphQL APIs every hour?

Here's the flow how this could work:

Create a repository containing a WunderGraph application
On each run, install the dependencies and call wunderctl generate to introspect all APIs and generate the configuration
Run npm test to run the tests with your test runner of choice
Run this workflow periodically and set up alerts for failed runs

Conclusion

Testing your GraphQL APIs is a great way to ensure that your APIs are correct and secure.
Using WunderGraph, you can test multiple GraphQL and REST APIs in a single test suite with minimal effort.
Thanks to the code-generation & code-first approach, we can spot errors early thanks to the type system and tsc compiler.

If you found this post interesting, you might want to follow me on Twitter or join our Discord Community and start a discussion.

Blog