HEY Turbolinks! RJS is dead ...

I am super excited about the release of the new product crafted by the Basecamp team, the HEY email service.

With all the @dhh teasing about the upgraded frontend stack, I was pumped to know how the new internals work, especially how the HEY app handles real-time updates and all the HTML mutations.

I want to share with you my thoughts about this subject and to be clear, it's only how I have understood what I have observed on the browser, nothing official.

The starting point

I tweeted yesterday about what I am seeing from the Ajax requests on the hey.com app

chadda chakib

@zimski1990

Interesting html mutation pattern seen on hey.com, the same pattern is used for updates over the socket
@dhh we need to know more about that !

09:25 AM - 19 Jun 2020

And this was retweeted by a basecamp guy, well known for his works on rails, Trix ... etc

Javan Makhmali

@javan

twitter.com/zimski1990/sta…

01:42 AM - 20 Jun 2020

chadda chakib @zimski1990
Interesting html mutation pattern seen on https://t.co/tX5zyCrhIH, the same pattern is used for updates over the socket @dhh we need to know more about that ! https://t.co/Y0Uheoo5fG

So I tell my self, maybe I should dig more to understand how this works.

Everything starts with an HTML fragment

<template data-page-update="remove#posting_140----"</template>

<template data-page-update="prepend#postings">
    <article
        id="posting_-----"
        class="posting"
        data-list-target="item"
        data-bulk-actions-target="row"
        data-identifier="------"
        data-sort-code="-------"
        data-box-freshness-target="row"
        data-account-id="----"
        data-account-purpose="home"
        data-box-kind="imbox"
        data-topic="true"
    >
        <div class="posting__body">
        ...
        </div>
    </article>
</template>

When I dig on the obfuscated JavaScript available on the app, I understand this mechanism:

1. Receiving data

The payload received from the socket or an Ajax call.
The payload is a pure HTML.

2. Processing the payload

This payload will be ingested and transformed into a valid HTML fragment document.

... document.createRange().createContextualFragment(playload)

When we have a valid html document, it's easier to process each tag and extract the needed informations:

... querySelectorAll("template[data-page-update]"

3. Commands to mutate the HTML

Each tag will contain the command to be executed.
This command is present on the data-page-update attribute.

Commands I have seen on the code:

append
prepend
replace
update
remove

The command follow this syntax: [command]#[html_element_id]

The <template..></template> tag can be empty, for commands like remove or containing an HTML inside to be inserted somewhere.

So if I follow all of this, we can see a cleaner version of the RJS used today on the rails app.

No need to write and render a custom JS code to mutate the dom.
With the current version, we were doing something like

$('#posting_1409180').remove();
$('#postings').prepend(<%= render... %>)

Now, we have a clean HTML that contains the command to mutate the DOM.

This follows the Stimulus principals to add some attributes to the HTML to get sparkles ✨.

The Content Template element

The documentation says that the <template> is not rendered and we need some js to render it.

We can imagine here to mix a standard HTML with this mutation template and render them on the page.
The js processor will pick and execute the mutation.

We can also stack all the mutations needed

<tempate mutation 1   />
<tempate mutation 2   >..</tempate>
<tempate mutation 3   >..</template>

So we will get a 100% HTML over the wire, no Javascript, no JSON, and a clever client-side HTML renderer to execute these mutations.

I think The main candidate to embrace this new mission is our beloved Turbolinks.

Turbolinks and sockets for a Realtime without hassle

Using this pattern, I suppose on the next version of turbo links we can tell it to subscribe to some cable channel and will render all of this HTML mutations

Rails helper on the backend ??

I hope also this will come with some helper to simply edit these template mutations.

Blog