Iterate quickly using the new --only-changed option
Simon Knott
Posted on August 21, 2024
Playwright v1.46 ships with a nifty new feature for local development.
By specifying the --only-changed
option, Playwright looks at uncommited changes and runs all affected test files.
If you've used Jest before, this is pretty similar to Jest's --onlyChanged
flag.
Let's see it in action. If you prefer video, watch the Demo in the release video. In this example, we have a repository without uncommitted changes:
// utils.ts
export const inputPlaceholder = 'What needs to be done?'
export const todoItemTestID = 'todo-item'
export const todo = 'buy milk'
// test.spec.ts
import { test, expect } from '@playwright/test'
import { inputPlaceholder, todo, todoItemTestID } from './utils.ts'
test('adding a todo', async ({ page }) => {
await page.goto('https://demo.playwright.dev/todomvc')
const inputField = page.getByPlaceholder(inputPlaceholder)
await inputField.fill(todo);
await inputField.press('Enter');
await expect(inputField).toBeEmpty();
await expect(page.getByTestId(todoItemTestID)).toHaveText(todo);
})
If we run a test with --only-changed
, we see that no tests are run:
$ npx playwright test --only-changed
Error: No tests found
Now let's make a change to one of the files, but not commit it yet:
// utils.ts
export const inputPlaceholder = 'What needs to be done?'
export const todoItemTestID = 'todo-item'
- export const todo = 'buy milk'
+ export const todo = 'make cappucino'
❯ npx playwright test
Running 1 test using 1 worker
✓ 1 [chromium] › test.spec.ts:4:5 › adding a todo (426ms)
1 passed (837ms)
Playwright found changes in utils.ts
, so it executed all test files that depend on utils.ts
. Wonderful!
By default, Playwright will compare against HEAD
to determine changed files.
But you can also specify a different Git revision to compare against: Using --only-changed=main
on a feature branch executes all tests that are affected by changes on the feature branch. And --only-changed=<commit sha>
executes all changes between HEAD
and the specified commit.
How do I use this?
--only-changed
is another tool in the toolbelt to help you iterate quickly. Here's five ideas for how you can integrate it into your workflow:
1. Local development: Use --only-changed=main
to quickly run affected tests as you make changes. This is especially useful when you're working on a feature and want to see the impact of your changes.
2. pre-commit hook: Running with --only-changed
in a pre-commit hook prevents failing tests from being commited. Here's an example hook you can use:
npx playwright test --only-changed
3. pre-push hook: As the name says, a pre-push hook runs before pushing. This prevents failing tests from invoking a costly CI run. Here's an example implementation:
while read local_ref local_sha remote_ref remote_sha
do
# deleting branch, nothing to check
if [ "$local_sha" = '0000000000000000000000000000000000000000' ]; then continue; fi
echo "Running Playwright for HEAD...$remote_sha"
npx playwright test --only-changed="$remote_sha"
done
4. Faster feedback in CI: Prior to a full run, use --only-changed
to get faster feedback on new failures. This is especially useful in large test suites, where running all tests can take a long time. Here's how that'd look in GitHub Actions:
...
- name: Install Playwright Browsers
run: npx playwright install --with-deps
+ - name: Run changed Playwright tests
+ run: npx playwright test --only-changed=${{ github.base_ref }}
- name: Run Playwright tests
run: npx playwright test
...
5. Bonus idea: If you're interested in --only-changed
, you might also be interested in Playwright's Watch mode that's available in the UI and in the VS Code Extension.
That's it! Let us know what you think and how you're using --only-changed
. Either in the comments here, in the Community Discord, or on LinkedIn! And have a wonderful day.
Posted on August 21, 2024
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.