Cypress is an integrated testing framework for web apps, and web components. It's popular due to it's ease of setup, and visual browser. In this blog I discuss parallel test running with the open source tool sorry cypress.

TL;DR

If you want to host the sorry cypress kit, then https://render.com/ makes it simple.

There is a sample repo here: https://github.com/simonireilly/render-sorry-cypress

Results

When hosting cypress in parallel for the default cypress test suite I performed a comparison on github actions:

It's really a great deal faster to invest in running the test suite in parallel even with just 20 test files, there is a huge benefit to cycle time.

Sorry cypress running in github actions

The speed benefits are certainly worthwhile, and the scalability unlocked by a coordinator like sorry cypress, over a configured parallel setup, means low configuration complexity.

Background

I have been using Cypress as a test runner since 2019, and enjoyed it's low barrier to entry.

It is probably well know that Cypress itself is not the fastest test runner, because it uses a full browser when running tests instead of other tools that might use a webkit only.

The advantages of Cypress are that it has a fast release cadence, adds new features frequently, and is intuitive.

To combat the slow test runtimes cypress provides a hosted director. The director issues batches of tests to each runner that joins the pool for that build id. It looks something like this:

Sorry cypress architecture, on the assumption that cypress themselves do something similar but at a much larger scale!

Running Sorry Cypress

There are some setup instructions for self hosting sorry cypress here.

I wanted to try hosting Sorry Cypress on render.com because:

• I have used render for multi-container apps before and it just makes sense.
• Render has a good integration with GitHub which helps streamline setup.
• The cost of running could be minute, something like:
  • $7/month + $0.25/month to run a mongoDB instance
  • $0/month for all other container running on demand; which works well for my use case
  • $7.25/month total; well isn't that nice, for 3 docker containers and a DB.

Render Setup

To run sorry cypress on render it's straight forward; here is a blueprint yaml file that will work for you:

The blueprint-spec is here, I recommend a read: render.com/docs/blueprint-spec

services:
  - type: web
    name: director
    env: docker
    healthCheckPath: /
    dockerfilePath: ./Dockerfile.director
    envVars:
      - key: DASHBOARD_URL
        fromService:
          name: dashboard
          type: web
          envVarKey: RENDER_EXTERNAL_URL
      - key: EXECUTION_DRIVER
        value: ../execution/mongo/driver
      - key: SCREENSHOTS_DRIVER
        value: ../screenshots/s3.driver
      - key: MONGODB_DATABASE
        value: sorry-cypress
      - key: MONGODB_URI
        fromService:
          name: mongo
          type: pserv
          envVarKey: MONGODB_URI

  - type: web
    name: dashboard
    env: docker
    healthCheckPath: /
    dockerfilePath: ./Dockerfile.dashboard
    envVars:
      - key: GRAPHQL_SCHemA_URL
        fromService:
          name: api
          type: web
          envVarKey: RENDER_EXTERNAL_URL
      - key: PORT
        value: 8080
      - key: CI_URL
        value: ""

  - type: web
    name: api
    env: docker
    dockerfilePath: ./Dockerfile.api
    envVars:
      - key: MONGODB_URI
        fromService:
          name: mongo
          type: pserv
          envVarKey: MONGODB_URI
      - key: MONGODB_DATABASE
        value: sorry-cypress

  - type: pserv
    name: mongo
    env: docker
    dockerfilePath: ./Dockerfile.mongo
    envVars:
      - key: MONGO_INITDB_ROOT_PASSWORD
        value: <password>
      - key: MONGO_INITDB_ROOT_USERNAME
        value: sorry-cypress
      - key: MONGODB_URI
        value: mongodb://sorry-cypress:<password>@<replace-with-pserv-address>:27017
    disk:
      name: data
      mountPath: /data/db
      sizeGB: 1

You will need to update the <password> before deploying the blueprint, and on first pass it will fail.

After you have mongo deployed you will need to edit the template setting <replace-with-pserv-address> to the network addressable name of your mongo instance:

The addressable endpoint for mongo is not known until deploy time; this would be fine if Sorry Cypress supported MONGO_HOST etc. but it expects a fully qualified URI, and render does not yet support env var interpolation

The benefit of render is that it will pick up on these changes, and notice the dependency chain between env var's exported from other services. It will re-deploy on each change, and reboot those services.

Running tests

I have an example runner setup to use github actions, which was shown in the early part of the blog.

The setup itself was very straight forward.

Sorry Cypress dashboard showing my example project.

The runs themselves are reported to sorry cypress from the test runners and appear inside the examples. I decided to use the git sha for my test runs since it will be unique all of the time:

Sorry Cypress example test run in the sorry cypress dashboard

Wrap up

I loved using sorry cypress, and hosting it on render was a lot easier than I expected.

Since render is setup to scale, I could see this being the most intuitive way to continue to use sorry cypress at a super low cost of $7.25/month; until my use case expands. At that point scaling up on render seems as easy as a few lines of yaml 👍