Iterate quickly using the new --only-changed option

skn0tt

Simon Knott

Posted on August 21, 2024

Iterate quickly using the new --only-changed option

Playwright v1.46 ships with a nifty new feature for local development.
By specifying the --only-changed option, Playwright looks at uncommited changes and runs all affected test files.

If you've used Jest before, this is pretty similar to Jest's --onlyChanged flag.

Let's see it in action. If you prefer video, watch the Demo in the release video. In this example, we have a repository without uncommitted changes:

// utils.ts
export const inputPlaceholder = 'What needs to be done?'
export const todoItemTestID = 'todo-item'
export const todo = 'buy milk'
Enter fullscreen mode Exit fullscreen mode
// test.spec.ts
import { test, expect } from '@playwright/test'
import { inputPlaceholder, todo, todoItemTestID } from './utils.ts'

test('adding a todo', async ({ page }) => {
    await page.goto('https://demo.playwright.dev/todomvc')

    const inputField = page.getByPlaceholder(inputPlaceholder)
    await inputField.fill(todo);
    await inputField.press('Enter');

    await expect(inputField).toBeEmpty();
    await expect(page.getByTestId(todoItemTestID)).toHaveText(todo);
})
Enter fullscreen mode Exit fullscreen mode

If we run a test with --only-changed, we see that no tests are run:

$ npx playwright test --only-changed
Error: No tests found
Enter fullscreen mode Exit fullscreen mode

Now let's make a change to one of the files, but not commit it yet:

// utils.ts
export const inputPlaceholder = 'What needs to be done?'
export const todoItemTestID = 'todo-item'
- export const todo = 'buy milk'
+ export const todo = 'make cappucino'
Enter fullscreen mode Exit fullscreen mode
❯ npx playwright test

Running 1 test using 1 worker

  ✓  1 [chromium] › test.spec.ts:4:5 › adding a todo (426ms)

  1 passed (837ms)
Enter fullscreen mode Exit fullscreen mode

Playwright found changes in utils.ts, so it executed all test files that depend on utils.ts. Wonderful!

By default, Playwright will compare against HEAD to determine changed files.
But you can also specify a different Git revision to compare against: Using --only-changed=main on a feature branch executes all tests that are affected by changes on the feature branch. And --only-changed=<commit sha> executes all changes between HEAD and the specified commit.

How do I use this?

--only-changed is another tool in the toolbelt to help you iterate quickly. Here's five ideas for how you can integrate it into your workflow:

1. Local development: Use --only-changed=main to quickly run affected tests as you make changes. This is especially useful when you're working on a feature and want to see the impact of your changes.

2. pre-commit hook: Running with --only-changed in a pre-commit hook prevents failing tests from being commited. Here's an example hook you can use:

npx playwright test --only-changed
Enter fullscreen mode Exit fullscreen mode

3. pre-push hook: As the name says, a pre-push hook runs before pushing. This prevents failing tests from invoking a costly CI run. Here's an example implementation:

while read local_ref local_sha remote_ref remote_sha
do
    # deleting branch, nothing to check
    if [ "$local_sha" = '0000000000000000000000000000000000000000' ]; then continue; fi

    echo "Running Playwright for HEAD...$remote_sha"
    npx playwright test --only-changed="$remote_sha"
done
Enter fullscreen mode Exit fullscreen mode

4. Faster feedback in CI: Prior to a full run, use --only-changed to get faster feedback on new failures. This is especially useful in large test suites, where running all tests can take a long time. Here's how that'd look in GitHub Actions:

...
    - name: Install Playwright Browsers
      run: npx playwright install --with-deps
+   - name: Run changed Playwright tests
+     run: npx playwright test --only-changed=${{ github.base_ref }}
    - name: Run Playwright tests
      run: npx playwright test
...
Enter fullscreen mode Exit fullscreen mode

5. Bonus idea: If you're interested in --only-changed, you might also be interested in Playwright's Watch mode that's available in the UI and in the VS Code Extension.

That's it! Let us know what you think and how you're using --only-changed. Either in the comments here, in the Community Discord, or on LinkedIn! And have a wonderful day.

💖 💪 🙅 🚩
skn0tt
Simon Knott

Posted on August 21, 2024

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Tagging your Playwright Tests
playwright Tagging your Playwright Tests

March 14, 2024

Debugging Playwright Tests in VS Code
playwright Debugging Playwright Tests in VS Code

November 22, 2023

Writing Playwright Tests in VS Code
playwright Writing Playwright Tests in VS Code

November 15, 2023

Introduction to Playwright
playwright Introduction to Playwright

November 2, 2023