Writing and debugging Cypress tests

I want to share the knowledge I and my team gained after implementing Cypress e2e tests in our project.

Intro

About a year ago we've (Visual Composer dev team) picked Cypress as our main tool for e2e tests. It was a new experience for the whole team as none of us was familiar with e2e testing. Over that period of time not only we've covered a large amount of functionality with tests but also we've learned quite a bit.

Recently I've made a small demo for our team of developers to summarise what we already know. The purpose of this demo was to:

Share knowledge and to establish common understanding and principles for e2e tests among all developers in our team

That inspired me to write an article.

Outline

Conventionally we can split the app testing process into two parts:

writing tests - when a developer actually writes the code
debugging tests - fixing issues when the test is failing

So that's what I'm going to share with you in a form of tips and solutions.

NOTE: Even though our product may be very specific I think these tips can be applicable to a wide variety of projects.

Writing tests

These tips helped me to write tests and to tackle issues faster and with ease.

1. Cypress syntax is similar to jQuery

If you know jQuery then understanding that Cypress commands are similar to the jQuery counterparts like selectors and chaining will make your Cypress learning curve much smoother.

E.g. the cy.get() command will return a collection of elements. Knowing that you'll be able to handle further chaining or resolve the possible error.

2. Cypress is lacking some features

Some of the functionality cannot be tested like hover effect or testing within and iframe.

The good part is that the Cypress team is aware of that and communicating to the community on possible solutions and workarounds.

3. Cypress resources

Continuing the thought from the previous point, I really like Cypress API documentation and their GitHub repo's Issues.

I was able to resolve almost every problem just by checking their docs or searching GitHub issues. Stackoverflow helped me as well. 😅

Overall I find the Cypress team quite engaging and willing to help.

4. Folder structure

We are sticking to the recommended folder and file structure in the official Cypress docs.

Since our project is quite large and multiple components that need to be tested are stored in multiple repositories, having a unified structure helps a lot.

5. Cypress strange behavior

In our case sometimes Cypress may render duplicate elements. I'm not sure if this is a project-specific issue, but I couldn't find any information on that topic.

The point is if a specific issue is occurring in your tests, you need to share that information across your team and prepare a solution for such cases.

Btw, the solution to our issue is since we're only checking the single element we're adding an index with a bracket notation to select the first element.

cy.get('.vce-row-content')[0]
  .children()
  .should('have.length', 3)

6. CSS values are computed values

When checking for CSS values it's important to understand that Cypress will compare your assertion to a computed CSS value. That is equal to the one you get when using the getComputedStyle() method.

Debugging

We separate debugging into two types.

Local
Pipeline

Local debugging

Usually, it's quite simple but sometimes developers get stuck and having a hard time resolving the issue. Even though the Cypress itself helps to troubleshoot.

1. Carefully read the error message

Developers are in a hurry and don't read the error message till the very end or read it superficially.

Sometimes it may contain a clue or a possible solution.

2. Additional data in the console

To retrieve additional data click on the step and the data will be outputted in the console.

3. Comment out commands

Comment out commands to get to the problem point faster. We have quite complex logic behind our tests, so they take quite a long time to run. In order to make the debugging process faster, I use that technique.

/* global describe, it, cy */

const ELEMENT_NAME = 'Text Block'

describe(ELEMENT_NAME, function () {
  it('Adds element to the page, checks automatically added elements, checks attributes', function () {
    cy.fixture('../fixtures/textBlock.json').then((settings) => {
      cy.createPage()
      cy.addElement(ELEMENT_NAME)

      // cy.setTinyMce({
      //   title: 'Content',
      //   text: settings.text,
      //   elementType: {
      //     name: settings.elementType.name
      //   },
      //   alignment: {
      //     name: settings.alignment.name
      //   }
      // })

      cy.setClassAndId(settings.customId, settings.customClass)

      // cy.setDO(settings.designOptions)

      cy.savePage()
      cy.viewPage()

      cy.get('.vce-text-block')
        .should('have.class', settings.customClass)
        .should('have.attr', 'id', settings.customId)

or go straight ahead to the exact page (our tests generate site pages in WordPress)

/* global describe, it, cy */

const ELEMENT_NAME = 'Text Block'

describe(ELEMENT_NAME, function () {
  it('Adds element to the page, checks automatically added elements, checks attributes', function () {
    cy.fixture('../fixtures/textBlock.json').then((settings) => {
      // cy.createPage()
      // cy.addElement(ELEMENT_NAME)

      // cy.setTinyMce({
      //   title: 'Content',
      //   text: settings.text,
      //   elementType: {
      //     name: settings.elementType.name
      //   },
      //   alignment: {
      //     name: settings.alignment.name
      //   }
      // })

      // cy.setClassAndId(settings.customId, settings.customClass)

      // cy.setDO(settings.designOptions)

      // cy.savePage()
      // cy.viewPage()

      cy.visit('http://localhost:8888/wp/auto-draft-4')

      cy.get('.vce-text-block')
        .should('have.class', settings.customClass)
        .should('have.attr', 'id', settings.customId)

Pipeline debugging

We use two different environments to run Cypress on the pipeline:

CircleCI on GitHub for our main project
Gitlab CI on GitLab for other projects

They both basically do the same stuff, the main difference is the interface.

On pipeline we use Docker image with:

Apache server
Pre-installed WordPress
Pre-installed theme
Pre-installed plugins

Debugging on the pipeline is essentially similar to a local.

1. Read the error message

Like in the Local debugging read the error, half of the times it will be enough. The image below is the error message from the GitLab CI terminal.

2. See artifacts (screenshot or video)

By default Cypress have screenshots enabled, but we also enabled video recording. To improve performance a little, we've enabled video recording only on failure. Video recording can be enabled in the cypress.json config file:

"video": true

Once the test is failed now we can check a screenshot or a video.

Artifacts on GitLab CI:

Artifacts on CircleCI:

3. Plugins for debugging

The following plugins can be used both locally and on the pipeline. We're saving up resources and using them rarely only for intricate cases. For the most part, it's enough with error messages and artifacts, but it's good to know that such plugins exist.

Saves the Cypress test command log as a JSON file if a test fails https://github.com/bahmutov/cypress-failed-log
A Cypress plugin that sends all logs that occur in the browser to stdout in the terminal https://github.com/flotwig/cypress-log-to-output

Inner docs

If your team or project has an inner knowledge base or even if you don't, start documenting. Over time there's just too much information piling in the heads of developers.

We've begun documenting howtos and practices on e2e testing. That way all developers now have a point of reference.

You don't have to scale to full-blown documentation at once. Start with a single document and a few code snippets. 😉

Conclusion

These are the techniques we use to write the e2e tests. I do hope that people can draw from our knowledge and improve their testing. Let us know what techniques and methods you are using by replying in the comments.

Check out our tests they are available in our GitHub repo.

Blog