Whatever happened to the test pyramid?
Daniel Irvine đłď¸âđ
Posted on February 6, 2020
Mike Cohnâs wonderful book Succeeding with Agile talks about the Test Automation Pyramid, shown in the image above.
This pyramid is beautifully explained by Martin Fowler and I certainly canât do even half as good a job as he can, so if youâre unfamiliar with the concept I suggest you read that post.
A key point is that unit tests should make up the bulk of your tests because they are cheaper to write and maintain, and because they are faster to run.
Recently though, I keep seeing and hearing from JavaScript folks who believe that unit testing is more trouble than itâs worth. There are smart, thoughtful front-end programmers who truly believe that end-to-end tests are the way to go.
So what happened to the humble unit test? Is the test pyramid still relevant?
Front-end frameworks make unit testing techniques less obvious
I think the main reason for lack of faith in the test pyramid is that front-end unit testing is fraught with danger. It can take a great deal of experience and confidence to be able to write a front-end unit test suite that is cheaper and easier to maintain than a set of UI-level tests.
Unit testing is proving harder and harder to do as we begin using frameworks that mesh together static data with behavior, like how React source files are a mixture of HTML and JavaScript.
Isnât the front-end just the UI tip of the pyramid?
No.
Modern front-end applications involve user workflow, receiving and sending data through network requests, handling session state and data validation. For applications that involve content creation, thereâs plenty of business logic to contend with, too.
All of these lend themselves well to unit testing.
If youâre working in a modern JavaScript codebase, you would do well to structure it with a standard hexagonal architecture. Then unit testing becomes straightforward, and the test automation pyramid begins to make sense again.
Knowing what not to test: the declarative nature of the front-end
Broswer-based development (i.e. âthe front-endâ) is a lot of HTML and CSS mixed in with some JavaScript. Thankfully the HTML and CSS is static data, and unit tests arenât generally meant for static data.
Unit tests really shine when youâre testing behavior, not simply repeating static information from the production codebase.
This applies to any framework you might be using, be it React or Vue or Svelte, or simply just plain JavaScript.
If you can create a very clean separation between your static data and your behaviour, then unit testing becomes painless.
Unfortunately, that separation isnât very easy to do. You can read about my way of doing it in React in this blog post.
HTML and CSS can easily be verified by visual regression and golden master tools
If youâve got a QA team, they are, without a doubt, interested in verifying that your static HTML and CSS does what itâs meant to.
They can use visual regression tools for this exact purpose.
Jest has a mechanism called snapshot testing which does just this, and itâs quite nice in that it can run on individual components in a component tree. Smaller units reduces the brittleness of these tests.
Unfortunately, these tests are often written in the same test suite as your unit tests, and snapshot testing is promoted as a kind of replacement for unit tests, even though the tests serve a different purpose (they donât test behavior) and arenât necessarily a good fit for a unit test framework like Jest.
My take on why the original pyramid isnât working
The assumption that unit tests are cheaper is no longer true simply because itâs harder to write unit tests.
Unit tests are only cheap if you have enough experience on your team to know how to write unit tests well.
What I observe from years of working with component-based frameworks is that people really struggle with unit tests.
If this is you, then donât be pressured (by people like me!) into thinking youâve got to start writing unit tests. Instead, the onus is on us (people like me!) to help simplify unit testing practice. Until then, stick with whatever form of testing brings you most value.
Posted on February 6, 2020
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related
February 11, 2020