Is abnormally tedious

Disclaimer: The opinion I formed is based solely on my personal experience. If you disagree (or even if you agree) with any or all of it, please feel free to comment and discuss it.

Automating this and automating that is mainly what we do in our day and age. Manual procedures, when they can be avoided without causing all kinds of havoc, are so very last decade. A natural fit for development, automation tests are all the rage. They have been for quite some time, but are now reaching the tipping point of becoming an essential part of every project, big or small.

And they can be, even for a small project, so many of them. So many in fact that without proper categorization their management would probably require a separate set of automation tests. Now, anyone who undertook the task of creating and managing a bunch of those tests will tell you about the pure madness categorization brings to the table.

This is my take on everything I read and tried out when it boils down to categorizing your tests neatly and efficiently.

Unit test, integration test, functional test.

Unit tests are the smallest and quickest tests you will have running. They will always test one very specific component. In an ideal world, where things are as they should, unit tests will be blazing fast and require no component to component interaction. Unit tests, again if done correctly, will render the debugger use needless. If a unit test fails, you know exactly what failed.

Integration tests will scope up a bit and include two or more components. Do they interact as they should, do they work when interacting? Such questions that boggle the mind are answered here, but the scope -even scaled up- does not go all out crazy to include a whole bunch of components. How many? Well, I was really not able to find a definitive limit. Let’s just keep it to a couple of them.

Functional tests, on the other hand, do go crazy. Since their purpose is to ensure an application function works as it should. Thus it is the least unusual for many more than two components to be included across the duration time of the test. Heavy use of debugging will occur once failed tests rear their ugly head since there is no way to easily detect the failed piece of code.

Acceptance test, end to end test.

If the above seems quite easy to grasp, here things get derailed fast. According to CodeceptJS, a family member of the rather widely used testing framework Codeception, end to end tests and acceptance tests are the same thing. That was a startling claim. I was not able to find anyone else backing up that point of view, I admit. They may exist, but I certainly was not able to locate them.

End to end tests are mostly about the testing duration as a direct result of all-inclusive testing. If for example, you want to test the user registration of any web application you will test each and every step (obviously testing both functionality and UI) until you conclude the registration process.

Acceptance, on the other hand, has more to do with whether or not the implemented functionality (that we already tested) is actually what the user wants. It is more of a business to end user and back to business again relationship. And it confused the world wide web out of me, at first.

It confused me because not everyone agrees on what they truly are. And even if they did, this distinction still seems rather cloudy especially when put next to the others that I mentioned above.

Google to the rescue.

As logic seemed to flee, Google decided to step in. Google testing blog is a great place to visit. Since they have to run and maintain a huge amount of tests, they always aim for simplicity over unnecessary complexity. It was there that I found a distinction that actually made sense. A distinction based on mostly concrete traits instead of abstraction and subjectiveness. Time.

Small tests are our unit tests. Each will run almost instantly, and it will retain all unit tests properties as already mentioned above.

Medium tests are our functional tests. They will run for more time than unit tests and will touch more than one components.

Large tests will be our end to end based on whole scenarios. These will include a large number of components and will -in turn- perform the slowest of them all.

And there you have it. Of course there will be more properties to classify among these three categories, and the time limits are not set in stone, but it all makes sense. So that is what I decided to follow when I set my tests apart. A system that everyone will simply understand.

Originally posted on Thanpa