Time to add some end to end tests for your next application. Today I have a new browser automation tool to share.

This is written from the perspective of the team heavily using JavaScript as the development language. I evaluated a few browser automation frameworks for end-to-end application testing and thought to share things the current landscape of tooling.

An End to end test is also known as functional-tests or happy-path tests which automates the sanity or spot checks before or after a software release.

I needed to pick one considering ease of development, long term maintainability, test framework integration. Let's explore a few interesting players out there.

1. The good old Selenium
2. The Chromium based browser automation API Puppeteer
3. The most hyped Cypress
4. Taiko - The new kid on the block (It's ok if you have never heard of this. I didn't too)

TL;DR; I ended up picking Taiko for its ❤️ beautiful API and it just works like magic!

1. Selenium ❌

Selenium was built in 2004. The major pain point of Selenium is its architecture.

A system that behaves differently in each run, also termed as flakiness is the major concern with Selenium with its layered architecture. Let's see why Selenium tests are flaky.

In the diagram, it is obvious that the commands have to pass through multiple layers. The drivers are built external to the browser which results in driving the browser "blind" with no feedback on whether the command has successfully been executed.

2. Puppeteer ❌

Fast forward from 2004, 14 years later, Google released v1.0 of Puppeteer.

One thing you will notice from the previous diagram is there are fewer layers between the controller and the browser. This is the key difference between the modern end to end testing frameworks.

Puppeteer is a Node library which provides a high-level API to control headless Chrome or Chromium over the DevTools Protocol.

This is powerful. It uses the DevTools protocol, the same low-level protocol which the Chrome Developer tools uses to interact with the browser.

A code example:

const browser = await puppeteer.launch();

const page = await browser.newPage();
await page.goto('https://example.com');

console.log(await page.content());
await page.screenshot({path: 'screenshot.png'});

await browser.close();

However, it is geared towards browser automation and the end to end web application testing is only one of them. This means you have to explicitly wait for a page to load, assert things you care about which will soon make writing and maintaining tests not so fun.

3. Cypress ❌

Cypress automates the browser with its own unique architecture. Cypress takes a less traveled and hard path to make runs the test from inside the browser and communicates the actions to an outside Node process which makes testing very capable.

It comes with electron baked in. Many times, I found very simple tests work in electron which they bundle by default however fails to run in Chromium or Chrome in a headless environment.

Rave reviews, Marketing, Podcast talks, Beautiful documentation, however, it does not live up to the high expectations you would expect.

I gave it enough energy and wanted this to work. Wanted this to work so badly. However, realized it has dangerous browser proxy mods which fail to set the redirections when run inside a corporate proxy environment and the most annoying limitation, if you are on one domain you cannot visit another domain.

Each test is limited to only visiting a single super-domain.

So if you have an application with a login page which is in another domain get ready to cry 😭. Cry so loud that Cypress team hears. I wish these change in the future. I gave up.

4. Taiko ✅

Taiko is from ThoughtWorks. The same place where Selenium was born. It shines ✨.

It shines really well with its ❤️ beautiful API.

Show me some code...

await openBrowser();
await goto("google.com");
await write("Dennis Mathew Philip");
await click("Google Search");

You will notice there is no waiting for an element to appear. No CSS selectors. It is smart to identify what input field.

✅ Beautiful documentation
✅ Integration with Jest and other popular testing frameworks
✅ No explicit wait needed. Less code
✅ Headless and CI friendly

It also has a REPL which makes the development a breeze.

A straightforward gif from the Taiko team:

I loved using Taiko. I am having fun writing end to end tests. I hope you will too.

To conclude...

End to end testing improves the confidence with which you release the software. We compared some of the free and open-source end to end testing solutions - Selenium, Puppeteer, Cypress and Taiko. The opinion written here is straight what I experienced. Consider your use-case closely and pick the right tool what works best for you. Browser automation tools is a lot of power. Use responsibly.

If your web application does not have at least one functional test, start today.

Please find my other writings here:

1. Automate your node dependency update
2. One side rounded rectangle using SVG
3. Visual Studio Code — Debug Mode
4. I want TypeScript to succeed.
5. What’s in my laptop?
6. CoffeeScript’s most loved feature soon in JavaScript