I was recently working on a project where I wanted to add some tests using Playwright to perform headless browser tests during the CI/CD pipeline.

Thankfully, my colleague Nitya has already written a blog post on how to do that, which you can read here.

This works just fine when the tests are running on the main branch, but I hit a snag with pull requests, because with Static Web Apps we get pre-production environments for pull requests and those are deployed with their own URLs.

Now we have a problem because in my tests I can’t just have:

test("basic test", async ({ page }) => {
    await page.goto("https://bit.ly/recipes-for-aj");
    await expect(page).toHaveTitle("Recipes 4 AJ");

    await page.locator("text=Tags").click();
});

Because that will always navigate to the production site! So, how can we solve this?

Finding the URL of a deployment

If you’ve looked into the logs of the deployment of a Static Web App you’ll have noticed that the URL is output there, whether it’s the URL with custom domain, or the pre-production environment URL on a PR, so this means that the GitHub Actions are aware of the URL.

Next stop, azure/static-web-apps-deploy to have a look at how the Action works. Alas it’s a Docker Action, which means we can’t see the internals of it, but that’s not a major problem because we can check out the actions.yaml and see the following:

outputs:
    static_web_app_url:
        description: "Url of the application"

Awesome! The Action will actually output the URL for us.

Using output across jobs

Following Nitya’s pattern, we’re going to create a new job in our workflow to run the Playwright tests:

jobs:
    build_and_deploy_job:
        # snip

    test:
        name: "Test site using Playwright"
        timeout-minutes: 60
        needs: build_and_deploy_job
        runs-on: ubuntu-20.04
        steps:
        - uses: actions/checkout@master
        - uses: actions/setup-node@v2
            with:
            node-version: '14.x'

        - name: Install dependencies
            run: |
                cd testing
                npm ci
                npx playwright install-deps
                npx playwright install
        - name: Run Playwright Tests
            continue-on-error: false
            working-directory: testing
            run: |
                npx playwright test --reporter=html --config playwright.config.js

We'll also update our test to use an environment variable to provide the URL, rather than having it embedded:

test("basic test", async ({ page }) => {
    await page.goto(process.env.SWA_URL);

    // Be assertive
});

To get that as an environment variable we have to first output it from the build_and_deploy_job:

jobs:
    build_and_deploy_job:
        if: github.event_name == 'push' || (github.event_name == 'pull_request' && github.event.action != 'closed')
        runs-on: ubuntu-latest
        name: Build and deploy website
        outputs:
            static_web_app_url: ${{ steps.swa.outputs.static_web_app_url }}
        steps:
        - uses: actions/checkout@v3
            with:
            submodules: true
        - name: Build And Deploy
            id: swa
            # snip

The important part is static_web_app_url: ${{ steps.swa.outputs.static_web_app_url }} in which were telling GitHub Actions that the step swa will have an output that we want to make an output of this job.

We can then use it in our test job like so:

test:
    name: "Test site using Playwright"
    timeout-minutes: 60
    needs: build_and_deploy_job
    runs-on: ubuntu-20.04
    env:
        SWA_URL: ${{ needs.build_and_deploy_job.outputs.static_web_app_url }}
    steps:
    # snip

The snippet ${{ needs.build_and_deploy_job.outputs.static_web_app_url }} tells GitHub Actions to look at the dependent job (needs.build_and_deploy_job) outputs and find the one we want and set it as an environment variable.

And just like that, you no longer need to have hard-coded URLs for your tests.

Conclusion

By leveraging output variables from GitHub Action steps and jobs we're able to simplify our GitHub workflows when it comes to doing something like automated tests using Playwright.

To show this in action I've created a PR for Nitya's sample application so you can see the changes that I made and how the GitHub Actions run now looks.