We can distinguish between several different types of testing. There are functional automated tests, in which we use Cypress or Selenium to test the correct operation of the functional parts of the application. Non-functional tests, on the other hand, include application performance testing.

Few people know that there is testing that is on the borderline between non-functional and functional testing. This kind of testing is called browser-based load tests. They involve measuring the functional readiness of an application based on metrics provided by the browser.

Real life scenario

To better understand this, let’s imagine a situation. We access a website and send dozens of requests to the application. Their response time does not indicate whether the site is ready for first use. Among other things, it testifies to how quickly the user is able to perform a given action on the site.

Browser-based load testing is all about measuring how quickly a user is able to start using an application. We do not test whether the functionality works and how it works. Instead, we measure the short times between the user entering the page and the first page elements loading.

Metrics

What specifically can we measure? Let’s discuss three main metrics.

The first of these is First Paint. This is the time between accessing the page and rendering the first pixels on the screen. We consider the first pixels to be anything that is visually different from what was on the screen before entering the site.

Another measure is First contentful paint. First is the time it takes users to start consuming content on a page. An indicator of this could be, for example, the rendering of any text, the first svg files or the rendering of the first pieces of content from the DOM.

The third measure is First Input Delay. It measures the time from when a user first interacts with a page (e.g. clicks a link or presses a button) until the browser is able to start processing event handlers in response to that interaction. What are page speed tests

Page Speed Insights

The most effective and widely used tool for retrieving metrics about a page is Page Speed Insights provided by Google. With its help, it is also possible to download other metrics. That said, it is not fully efficient. It takes a lot of time to analyze, which, with several hundred runs, can drag on for hours.

Measuring Page Speed Insights in k6 Testing

A performance testing tool called K6 allows you to write extensions to it. One such extension includes xk6-playwright. Developers of the extension based on the well-known e2e test tool created its implementation in K6 and extended it with additional functionality not available in the base version of playwright.

The additional functionalities are the metrics just discussed. Moreover, the extension implements an additional meter called Time to Minimally Interactive. With it, the time of the first input element (input) on the page is checked. It is also a recommended way to measure the page readiness time.

The xk6 package that we will use to run the tool is installed with the command xk6 build --output xk6-playwright.exe --with github.com/wosp-io/xk6-playwright. This will create a package through which we can run our test scenario. Let’s try to create a simple test in which we implement reading and displaying metrics in the standard output.

import pw from 'k6/x/playwright';
import { Trend } from 'k6/metrics';


export const options = {
  duration: '30s',
  vus: 1
};

export default function () {
  pw.launch()
  pw.newPage()
  pw.goto("https://app.k6.io/account/login")

  console.log(pw.firstPaint())
  console.log(pw.firstContentfulPaint())
  console.log(pw.timeToMinimallyInteractive())
  console.log(pw.firstInputDelay())
}

With several hundred runs, reading the data in the console can be hard to analyze. So let’s create a trend that will hold the aggregated results and then display them in a summary.

import pw from 'k6/x/playwright';
import { Trend } from 'k6/metrics';


export const options = {
  duration: '30s',
  vus: 1,
};

const firstPaint = new Trend("first_paint")
const firstContentfulPaint = new Trend("first_contentful_paint")
const timeToMinimallyInteractive = new Trend("time_to_minimally_interactive")
const firstInputDelay = new Trend("first_input_delay")

export default function () {
  pw.launch()
  pw.newPage()
  pw.goto("https://app.k6.io/account/login")

  firstPaint.add(pw.firstPaint())
  firstContentfulPaint.add(pw.firstContentfulPaint())
 timeToMinimallyInteractive.add(pw.timeToMinimallyInteractive())
  firstInputDelay.add(pw.firstInputDelay())
}

What remains to be done is to check whether the results agree with those we would like to achieve. We will use the threshold functionality offered by k6 to do this. For First Paint and First Contentful Paint, the optimal value should be below 1.8 seconds. On the other hand, for Time to minimally interactive and first input delay, the time should be below 100ms.

import pw from 'k6/x/playwright';
import { Trend } from 'k6/metrics';


export const options = {
  duration: '30s',
  vus: 1,
  thresholds: {
    first_paint: ['p(95)<1800'],
    first_contentful_paint: ['p(95)<1800'],
    time_to_minimally_interactive: ['p(95)<100'],
    first_input_delay: ['p(95)<100']
  }
};

const firstPaint = new Trend("first_paint")
const firstContentfulPaint = new Trend("first_contentful_paint")
const timeToMinimallyInteractive = new Trend("time_to_minimally_interactive")
const firstInputDelay = new Trend("first_input_delay")

export default function () {
  pw.launch()
  pw.newPage()
  pw.goto("https://app.k6.io/account/login")

  firstPaint.add(pw.firstPaint())
  firstContentfulPaint.add(pw.firstContentfulPaint())
 timeToMinimallyInteractive.add(pw.timeToMinimallyInteractive())
  firstInputDelay.add(pw.firstInputDelay())
}