custom elements and streamed HTML: Make CSR Great Again

This is a report for a recent experimental demonstrating project we are working on. However, the techniques used in the post is framework neutral.

Why are CSRs not great ?

Server side rendering, aka SSR, and static site generation, aka SSG, is now the standard de facto of web app practice. You basically ship a web page that "just works" to the browser, and the browser can just render it, without extra downloads and execution. That is just the "correct answer" for the browser side.

However the execution has to happen somewhere. In SSR, it happens on the server. If we take the server side execution time into consideration, the performance benefits of going SSR would not seem so promising. you cache it for later use so it won't run on every single request, but server resources are more precious, because you are not serving only one user. One single server side re-rendering execution may delay all concurrent clients, it can be, let's say, hundreds based on the server load.

In SSG, it happens when you build it. It is good if you are not rebuilding it again and again.

But somehow that is still better than CSR, even for quickly changing realtime data so SSG is completely out. The key point here is the total latency, of which the most significant factor is RTT.
SSR is guaranteed 1-RTT for first time and later, and the result can be cached. CSR is usually 2-RTT for first time, or even more, and will always re-render. SSG, with carefully designed caching strategy, is 1-RTT for first time and 0-RTT later since it is static, without re-renders.

Of course the benefit of going SSR or SSG is really not only about RTT, they are great in many aspects. I really love this article by Ryan on how great SSR is.

If it will anyway re-render, which is not necessarily a bad thing, is it possible to make a dynamic page (let's just assume the page may randomly change every single second) 1-RTT but CSR ?

Only if the data is sent along with the document, but not rendered. It is indeed somewhere in between CSR and SSR, but at least you are definitely not paying huge server side computational resources for that.

What custom elements / web components can do ?

It can be easily explained by a example:

<html lang="en">

<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <link rel="shortcut icon" href="assets/favicon.svg" type="image/svg+xml">
  <script src="/components/streamReceiver.js" type="module" async></script>
  <style>
    stream-data {
      display: none
    }
  </style>
</head>

<body>
  <stream-receiver data="stream-data"></stream-receiver>
  <stream-data data-text="0">0</stream-data>
  <stream-data data-text="1">1</stream-data>
  <stream-data data-text="2">2</stream-data>
  <stream-data data-text="3">3</stream-data>
  <stream-data data-text="4">4</stream-data>
  <stream-data data-text="5">5</stream-data>
  <stream-data data-text="6">6</stream-data>
</body>

</html>

There is a custom element stream-receiver introduced in /components/streamReceiver.js with an attribute data="stream-data". The connectedCallback is implemented as such:

    constructor() {
      super()
      const dataTag = this.getAttribute('data')?.toUpperCase()
      if (!dataTag) throw new TypeError('no dataTag')
      this.dataTag = dataTag
      this.shadow = this.attachShadow({ mode: 'open' })
    }
    connectedCallback() {
      const data = document.querySelectorAll(this.dataTag)
      for (const element of data) {
        this.onData(element)
      }
    }

The server could just embed the data in side the data-text attribute of the stream-data element, and the component could access those data with element.dataset.text.

So, 1-RTT dynamic page, with CSR.

Wait, this is not great at all.

So now we have streamed HTML, and it works great with the code above, with just a little bit tweak of the component.

    constructor() {
      super()
      const dataTag = this.getAttribute('data')?.toUpperCase()
      if (!dataTag) throw new Error('go full CSR')
      this.dataTag = dataTag
      this.shadow = this.attachShadow({ mode: 'open' })

      const p = document.createElement('p')
      p.textContent = 'Make CSR great again'
      this.shadow.appendChild(p)
    }
    connectedCallback() {
      const onMutation: MutationCallback = record => {
        for (const mutation of record) {
          if (mutation.type === 'childList') {
            const dataList = mutation.addedNodes
            for (const data of dataList) {
              if (data instanceof HTMLElement && data.tagName === this.dataTag) {
                this.onData(data)
              }
            }
          }
        }
      }
      const observer = new MutationObserver(onMutation)
      observer.observe(document.body, { childList: true })
      const data = document.querySelectorAll(this.dataTag)
      for (const element of data) {
        this.onData(element)
      }
    }
  }

Now we use a MutationObserver to observe the children of document.body. Hopefully we should only see stream-data since all the real UI changes go inside the component.

Streamed HTML, but how ?

I said this post is framework neutral, but for this part the code is for cloudflare "edge runtime". It is trivial to rewrite it for other platforms. It is server side so it has to be on some platform.

export const onRequest: PagesFunction = async ctx => {
  const document = new Document({
    component: {
      tag: 'stream-receiver',
      src: '/components/streamReceiver.js'
    },
    lang: 'en',
    dataTag: 'stream-data',
    favicon: { href: 'assets/favicon.svg', type: 'image/svg+xml' }
  })
  ctx.waitUntil(writeDocument(document))
  return document.response
}

The entry point is simple. Let's move to how that Document looks like.

export class Document {
  static favicon(href: string, type: string) {
    return `<link rel="shortcut icon" href="${encodeURI(href)}" type="${type}">`
  }
  static scriptLine(src: string) {
    return `<script src="${src}" type="module" async></script>`
  }
  static componentLine(tag: string, dataTag: string) {
    return `<${tag} data=${dataTag}></${tag}>`
  }
  static styleLine(dataTag: string) {
    return `<style>${dataTag} {display: none}</style>`
  }
  static opening({ component, lang, favicon, dataTag }: {
    component: { tag: string, src: string },
    lang?: string
    favicon?: { href: string, type: string }
    dataTag: string
  }) {
    return '<!DOCTYPE html>' +
      (lang ? `<html lang="${this.escape(lang)}">` : '<html>') +
      '<meta charset="UTF-8">' +
      '<meta name="viewport" content="width=device-width, initial-scale=1.0">' +
      (favicon ? this.favicon(favicon.href, favicon.type) : '') +
      this.scriptLine(component.src) +
      this.styleLine(dataTag) +
      '</head><body>' +
      this.componentLine(component.tag, dataTag)
  }
}

Let's begin with some helper functions. We are just playing with strings here to assemble what should be sent immediately. This is what the opening function would return:

<html lang="en">

<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <link rel="shortcut icon" href="assets/favicon.svg" type="image/svg+xml">
  <script src="/components/streamReceiver.js" type="module" async></script>
  <style>
    stream-data {
      display: none
    }
  </style>
</head>

<body>
  <stream-receiver data="stream-data"></stream-receiver>

And how we actually send it:

  readonly response: Response
  readonly dataTag: string
  private readonly writer: WritableStreamDefaultWriter<ArrayBuffer>
  private readonly encoder: TextEncoder
  private waitUntil?: Promise<void>

  private write(v: string) {
    this.waitUntil = (this.waitUntil || Promise.resolve()).then(
      () => this.writer.write(this.encoder.encode(v))
    )
    return this.waitUntil
  }

  constructor(options: {
    component: { tag: string, src: string },
    lang?: string
    favicon?: { href: string, type: string },
    dataTag: string
  }) {
    options.dataTag = options.dataTag
    this.dataTag = options.dataTag

    const { writable, readable } = new IdentityTransformStream()
    this.response = new Response(readable, {
      headers: { 'content-type': 'text/html' }
    })
    this.writer = writable.getWriter()
    this.encoder = new TextEncoder()
    this.write(Document.opening(options))
  }

A readable stream is used to create the response, and the writer can be used later to write the data.

Here is how the data is written:

const writeDocument = async (document: Document) => {
  for (let i = 0; i < 20; i++) {
    if (Math.random() > 0.5) await new Promise(cb => setTimeout(cb, 1000))
    document.write(`<stream-data data-text="${i}">${i.toString()}</stream-data>`)
  }
  document.close()
}

It is pretty much just writing the plain data, but it can be more complicated and provide structural data via the element itself, or it could just pass the json data to data-text. It it nothing like "rendering" though.

What's the good part of this technique ?

The opening lines reach the browser super fast. They are sent immediately upon request received. It leaves no overhead in terms of "TTFB". On a cloudflare pages deploy, browser starts to receive the document in 30ms, and the script in 60ms.
Nearly no computation on server side, compared with a full rendering. The average CPU time per request is 3ms.
JavaScript and data downloads happen in parallel.
The custom element runs on itself. It can render an interactive UI with or without the partially downloaded data before the document itself is complete: and the data would keep downloading during the execution.
Rendering is under full control of the component. Aggressive lazy loading can be easily applied and it can freely decide how the data is used with no overhead, based on client side conditions.
The download size of the document is smaller because plain json is usually smaller than the rendered html.
window.stop can abort the data download process.
The component script can be aggressively cached. The document itself can also be cached, but it is expected to update frequently.
The data is guaranteed fresh so JavaScript won't need another data request.
The document stream can be persistent so it can be reused to push updates.
Search engines can see the data without JavaScript. Or they can execute it and know how the page really looks like.

And what's the bad parts ?

It may not actually be a stream, especially if the user is behind a proxy. The server must be able to detect that, or it will be a disaster if the document is reused for updates.
It requires client side JavaScript support, as a normal custom element.
Always re-render on client side refreshes. It is not a good solution in terms of energy and client side resource efficiency, compared with SSG. But energy efficiency can be complicated.
window.stop is not very convenient.

Q&A

Why an empty custom element like that ?
- To support multiple custom elements on the same page.
- But yes it is trivial to put stream-data inside stream-receiver: remember to use a shadow root and prepare a default style in case the element is rendered before the custom element is ready. You are probably safe if you keep the receiver minimal.
So just inline the receiver ?
- It will block the rest of the stream (but is it really an issue ?). Good idea if it is really small.
Let's make the receiver universal.
- It depends on how you would define "universal". It is more practical to have multiple less universal versions to choose from.
That class Document seems not very useful.
- Feel free to enhance it. It is made minimal on purpose to show the idea.
Reuse the document connection for updates ?
- Not very practical on cloudflare, AWS, and probably most serverless platforms, but trivial on node.js or nginx.
- And if your favicon is a spinning circle.

Blog