Server Side Rendering a Blog with Web Components

This blog post supports a Youtube Livestream scheduled for Wednesday 4/19 at 12pm EST / 9am PST. You can watch the livestream here on Youtube.

Introduction

It has never been easier to server side render a website. Years ago it took server side technologies like PHP and Java, requiring web developers to learn another language besides JavaScript. With the introduction of Node.js, web developers got a JavaScript runtime on the server and tools such as Express that could handle HTTP requests and return HTML to the client. Meta-frameworks for server side rendering sprang up that supported popular libraries and frameworks like React, Angular, Vue, or Svelte.

Meta-frameworks had to consider React a first-class citizen because of how React utilizes Virtual DOM, an abstraction around DOM that enables the library to "diff" changes and update the view. React and Virtual DOM promised efficiency, but what we got was a toolchain that made the learning curve for web development more difficult. The ecosystem surrounding JavaScript libraries promised a boost in developer experience. What we got was dependency hell.

Hydration became part of the vocabulary of several web developers in recent years. Hydration being the process of using client-side JavaScript to add state and interactivity to server-rendered HTML. React popularized many of the concepts web developers think of when we say "server-side rendering" today.

What if I told you there was a way to server-side render HTML with less tooling, using a syntax that you don't need to learn another library to code? The solution just uses HTML, CSS, and JavaScript, with a bit of library code on the server. Would that be enticing?

Shameless plug. I'm Stephen Belovarich, the author of FullStack Web Components, a book about coding UI libraries with custom elements. If you like what you are reading in thispost consider purchasing a copy of my book Fullstack Web Components available at newline.co. Details following this post.

In this guide, I'll demonstrate how to server-side render autonomous custom elements using only browser specifications and Express middleware. In the middleware, I'll show you how to work with an open source package developed by Google to server-side render Declarative Shadow DOM templates. @lit-labs/ssr is a library package under active development by the team that maintains Lit, a popular library for developing custom elements. @lit-labs/ssr is part of the Lit Labs family of experimental packages. Even though the package is in "experimental" status, the core offering is quite stable for use with "vanilla" custom elements.

You can render custom elements today server-side with @lit-labs/ssr by binding the render function exported by the package to Express middleware. @lit-labs/ssr supports rendering custom elements that extend from LitElement first and foremost, although LitElement itself is based off web standards, the class is extended from HTMLElement. @lit-labs/ssr relies on a lit-html template to render content server-side, but lit-html happens to be compatible with another web standard named Declarative Shadow DOM.

Declarative Shadow DOM

Declarative Shadow DOM is the web specification what makes server-side rendering custom elements possible. Prior to this standard, the only way to develop custom elements was imperatively. You had to define a class and inside of the constructor construct a shadow root. In the below example, we have a custom element named AppCard that imperatively constructs a shadow root. The benefit of a shadow root is that we gain encapsulation. CSS and HTML defined in the context of the shadow root can't leak out into the rest of the page. Styling and template remains scoped to the instance of the custom element.

class AppCard extends HTMLElement {
  constructor() {
    super();
    if (!this.shadowRoot) {
      const shadowRoot = this.attachShadow({ mode: 'open' });
      const template = document.createElement('template');
      template.innerHTML = `
      <style>
      ${styles}
      </style>
      <header>
        <slot name="header"></slot>
      </header>
      <section>
        <slot name="content"></slot>
      </section>
      <footer>
        <slot name="footer"></slot>
      </footer>
      `;
      shadowRoot.appendChild(template.content.cloneNode(true));
    }
  }
}

customElements.define('app-card', AppCard);

The above example demonstrates how an autonomous custom element named AppCard declares it's shadow root imperatively in the constructor.

Declarative Shadow DOM allows you to define the same template for the custom element declaratively. Here is an example of the same shadow root for AppCard defined with Declarative Shadow DOM.

<app-card>
  <template shadowrootmode="open"> 
    <style>
      ${styles}
    </style>
    <header>
      <slot name="header"></slot>
    </header>
    <section>
      <slot name="content"></slot>
    </section>
    <footer>
      <slot name="footer"></slot>
    </footer> 
  </template>
  <img slot="header" src="${thumbnail}" alt="${alt}" />
  <h2 slot="header">${headline}</h2>
  <p slot="content">${content}</p>
  <a href="/post/${link}" slot="footer">Read Post</a>
</app-card>

Declarative Shadow DOM introduces the shadowrootmode attribute to HTML templates. This attribute is detected by the HTML parser and applied as the shadow root of it's parent element (<app-card>). The above example uses template slots to dynamically inject content into a custom element template. With Declarative Shadow DOM, any HTML defined outside of the HTML template is considered "Light DOM" that can be projected through the slot to "Shadow DOM". The usage of ${} syntax is merely for the example. This isn't some kind of data-binding technique. Since the template is now defined declaratively, you can reduce the definition to a String. ES2015 template strings are well suited for this purpose. In the demo, we'll use template strings to define each component's template declaratively using the Declarative Shadow DOM specification.

But wait? If the component's template is reduced to a string how do you inject interactivity into the component client-side? You still have to define the component imperatively for the client, but since Shadow DOM is already instantiated (the browser already parsed the Declarative Shadow DOM template), you no longer need to instantiate the template. You may still imperatively instantiate Shadow DOM if the shadow root doesn't already exist.

class AppCard extends HTMLElement {
  constructor() {
    super();
    if (!this.shadowRoot) {
      // instantiate the template imperatively
      // if a shadow root doesn't exist 
    }
    ...

Optionally, you can hydrate the component client-side differently when a shadow root is detected.

class AppCard extends HTMLElement {
  constructor() {
    super();
    if (this.shadowRoot) {
      // bind event listeners here
      // or handle other client-side interactivity
    }
    ...

The architects in the crowd may notice one flaw with this approach. Won't the same template need to defined twice? Once for Declarative Shadow DOM and a second time for the declaration in the custom element's constructor. Sure, but we can mitigate this by using ES2015 template strings. By implementing the template through composition, we can inject the template typically defined imperatively into the other defined declaratively. We'll make sure to reuse partial templates in each custom element developed in this workshop.

What You Will Build

In this workshop, you'll server-side render four custom elements necessary to display a blog:

AppCard displays a card
AppHeader displays the site header
MainView displays the site header and several cards
PostView displays the site header and a blog post

Acceptance Criteria

The blog should have two routes. One that displays a list of the latest posts, another that displays the content of a single post.

When the user visits http://localhost:4444 the user should view the site header and several cards (MainView).
When the user visits http://localhost:4444/post/:slug the user should view the site header and blog post content (PostView). The route includes a variable "slug" which is dynamically supported by the blog post.

Project Structure

The workspace is a monorepo consisting of 4 project directories:

client: Custom elements rendered client and server-side
server: Express server that handles server-side rendering
shim: Custom shim for browser specifications not found in Node.js, provided by Lit
style: Global styles for the blog site

Lerna and Nx handle building the project, while nodemon handles watching for changes and rebuilding the project.

The project is mainly coded with TypeScript. You'll be developing mostly server-side in this workshop which maybe a change for some people. When you run console.log this log will happen on the server, not in the browser, for instance.

For the workshop, you'll focus primarily on a single file found at /packages/server/src/middleware/ssr.ts. This file contains the middleware that handles server-side rendering. For the remainder of the workshop, you'll edit custom elements found in packages/client/src/. Each file includes some boilerplate to bootstrap the experience.

Architecture

In this workshop you'll develop methods for asynchronously rendering Declarative Shadow DOM templates. Each view is mapped to a Route. Both Route listed in the acceptance criteria are defined in this file: packages/client/src/routes.ts.

export const routes: Routes = [
  {
    path: "/",
    component: "main",
    tag: "main-view",
    template: mainTemplate,
  },
  {
    pathMatch: /\/post\/([a-zA-Z0-9-]*)/,
    component: "post",
    tag: "post-view",
    template: postTemplate,
  },
];

We need a static definition of the routes somewhere. Putting the definition in an Array exported from a file is an opinion. Some meta-frameworks obfuscate this definition with the name of directories parsed at build or runtime. In the middleware, we'll reference this Array to check if a route exists at the path the user is requesting the route. Above the routes are defined with an identifier: path. path: "/", matches the root, i.e. http://localhost:4444. The second example uses pathMatch instead. The route used to display each post is dynamic, it should display a blog post by slug. Each route also corresponds to a template, which we'll define in each "view" file as a function that returns a template string. An example of a template is below.

export const mainTemplate = () => `<style>
    ${styles}
  </style>
  <div class="container">
  <!-- put content here -->
  </div>`;

During the workshop, we'll display a static route, but soon after make each view dependent on API requests. The JSON returned from local REST API endpoints will be used as a model for the view. We'll export a named function called fetchModel from each view file that fetches the data and returns the model. An example of this is below.

function fetchModel(): Promise<DataModel> {
  return Promise.all([
    fetch("http://localhost:4444/api/meta"),
    fetch("http://localhost:4444/api/posts"),
  ])
    .then((responses) => Promise.all(responses.map((res) => res.json())))
    .then((jsonResponses) => {
      const meta = jsonResponses[0];
      const posts = jsonResponses[1].posts;
      return {
        meta,
        posts,
      };
    });
}

In the above example, two calls to fetch request the metadata for the site and an Array of recent blog posts from two different API endpoints. The responses are mapped to the model needed to display the view.

A description of each API endpoint is below, but first let's look at the flow of information.

Markdown -> JSON w/ embedded Markdown -> HTML -> Declarative Shadow DOM Template

Blog posts are stored in markdown format in the directory packages/server/data/posts/. Two API endpoints (/api/posts and /api/post/:slug) fetch the markdown from each file and return that markdown in the format of a Post. The type definition of a Post is as-follows:

export type Post = {
  content: string;
  slug: string;
  title: string;
  thumbnail: string;
  author: string;
  excerpt: string;
};

Another endpoint handles metadata for the entire page. The interface for the data returned by this endpoint is simplified for the workshop and could be expanded for SEO purposes.

export type Meta = {
  author: string;
  title: string;
};

During the workshop, you'll rely on three local API endpoints to fetch the metadata of the site and the data associated with each blog post. A description of each endpoint is below.

http://localhost:4444/api/meta returns the metadata necessary to display the site header in JSON format.

http://localhost:4444/api/post/:slug returns a single blog post by it's "slug", a string delineated by - in JSON format.

http://localhost:4444/api/posts returns an array of blog posts in JSON format.

You'll make requests to these endpoints and use the JSON responses to populate the content for each Declarative Shadow DOM template. In each file that requires data, a type definition for Meta and Post schema is already provided in packages/server/src/db/index.ts. These type definitions are imported into relevant files to ease in development.

In addition to the three provided local endpoints, you'll be making a request to the Github API to parse the markdown returned from the included blog post files. This API is necessary because it provides the simplest way to parse code snippets found in markdown files and convert them to HTML.

Getting Started

You'll need a GitHub account. You can use GitHub to sign-in to StackBlitz and later, you'll need GitHub to generate a token.

If you don't already have a GitHub account, signup for Github here.

If you want to follow along with this tutorial, fork the StackBlitz or Github repo.

When using Stackblitz, the development environment will immediately install dependencies and load an embedded browser. VS Code is available in browser for development. Stackblitz works best in Google Chrome.

If using Github, fork the repository and clone the repo locally. Run npm install and npm run dev. Begin coding in your IDE of choice.

Important Note About Tokens

During the workshop, we'll be using the Github API known as Octokit to generate client-side HTML from Markdown for each blog post. If you're using Stackblitz, an API token is provided for the workshop but will be revoked soon after. If you've cloned the repo or the token is revoked, login to GitHub and generate a new token on Github for use in the workshop.

Never store tokens statically in code. The only reason the token is injected this way for the workshop is to bootstrap making requests with Octokit.

Important Note About Support

If you are following along with StackBlitz, make sure you are using Google Chrome.

Examples included in this workshop will work in every mainstream browser except Firefox. The code should work in Google Chrome, Microsoft Edge, and Apple Safari.

Although Mozilla has signaled support for implementing Declarative Shadow DOM, the browser vender has yet to provide support in a stable release. A lightweight ponyfill is available to address the lack of support in Firefox.

Support for Declarative Shadow DOM is subject to change in Firefox. I have faith the standard will become cross-browser compatible in the near future because Mozilla recently changed it's position on Declarative Shadow DOM. I've been using the ponyfill for over a year without issues in browsers that lack support for the standard.

Server Side Rendering Custom Elements with @lit-labs/ssr

Let's get started coding, shall we?

The first task is declaring a Declarative Shadow DOM template for the component named MainView found in packages/client/src/view/main/index.ts. This view will ultimately replace the boilerplate currently displaying "Web Component Blog Starter" in the browser at the path http://localhost:4444/.

Supporting Declarative Shadow DOM in MainView

Open packages/client/src/view/main/index.ts in your IDE and find the Shadow DOM template that is defined imperatively in the constructor of the MainView class. We're going to declare this template instead as an ES2015 template string returned by a function named shadowTemplate. Cut and paste the current template into the template string returned by shadowTemplate.

const shadowTemplate = () => html`<style>
    ${styles}
  </style>
  <div class="post-container">
    <app-header></app-header>
  </div>`;

For your convenience, html is imported into this file, which you can use to tag the template literal. This html shouldn't be confused with the function exported from lit-html. It's a custom implementation of a tagged template literal that enables syntax highlighting of the string, if you have that enabled in your IDE.

If you opted for GitHub and cloned the repo locally, you can enable this VSCode extension that offers syntax highlighting of HTML and CSS inside tagged template literals. That's really the only purpose of html for now, although you could exploit this function and parse the template string in different ways if you wanted.

Update the imperative logic to call shadowTemplate which should return the same String as before.

template.innerHTML = shadowTemplate();

In an effort to reuse the template, we can pass it to the next function we'll declare named template. The format is the same, although this time the template normally declared imperatively is encapsulated by the HTML tag associated with the component (<main-view>) and a HTML template with the attribute shadowrootmode set to open. Inject the shadowTemplate() we defined earlier inside of the <template> tags.

const template = () => html`<main-view>
  <template shadowrootmode="open">
    ${shadowTemplate()}
  </template>
  </main-view>`;

You just declared your first Declarative Shadow DOM template. Some benefits of this approach is that we can reuse the typical shadow root template declared imperatively and since we're using function, inject arguments that can be passed to the ES2015 template string. This will come in handy when we want to populate the template with data returned from API endpoints.

Finally, be sure to export the template from the file. More on why we're exporting the MainView class and template function later.

export { MainView, template };

This is a good start, but we'll run into an issue if we keep the code as-is. While it is possible to reuse template partials, the above example is kind of an exaggeration. You'll rarely be able to inject the entire shadow root into the Declarative Shadow DOM template like we did, especially when there are child custom elements that also need to be rendered server-side with Declarative Shadow DOM. In the above example, <app-header> can't currently be rendered on the server because we haven't declared that component's template with Declarative Shadow DOM. Let's do that now.

Supporting Declarative Shadow DOM in Header

To declare a new template to server-side render the AppHeader component, open packages/client/src/component/header/Header.ts.

Just like in the last example, cut and paste the template that is defined imperatively in the AppHeader constructor into a new function named shadowTemplate that returns the same string. This time, inject a single argument into the function that destructs an Object with two properties: styles and title and pass those to the template.

const shadowTemplate = ({ styles, title }) => html`
  <style>
    ${styles}
  </style>
  <h1>${title}</h1>
`;

Call shadowTemplate with the first argument, setting the two properties that are currently found globally in the file: styles and title

template.innerHTML = shadowTemplate({ styles, title });

It's quite alright to define arguments in this way that later populate the template. It will offer some flexibility later on when we need to map the response of an API endpoint to what the template function expects. To declare the Declarative Shadow DOM template, define a new function named template, this time encapsulating the call to shadowTemplate with the <app-header> tag and HTML template.

const template = ({ styles, title }) => html`<app-header>
  <template shadowrootmode="open">
  ${shadowTemplate({ styles, title })}
  </template>
</app-header>`;

We can reuse the shadow root template used declaratively in this case because AppHeader is a leaf node, that is to say, it has no direct descendants that need to also be rendered server-side with Declarative Shadow DOM.

Finally, export all the necessary parts in MainView.

export { styles, title, template, AppHeader };

Updating MainView with the Declarative Header

Open packages/client/src/view/main/index.ts again and import the parts from Header.js needed to render the template this time renamed as appHeaderTemplate. Notice how styles as appHeaderStyles is used with as so they don't conflict with this local styles declaration in packages/client/src/view/main/index.ts.

import {
  styles as appHeaderStyles,
  template as appHeaderTemplate,
  title,
  AppHeader,
} from '../../component/header/Header.js';

Update the template function, replacing the shadowTemplate call with appHeaderTemplate, being sure to pass in the styles and title.

const template = () => html`<main-view>
  <template shadowrootmode="open">
    ${appHeaderTemplate({ styles: appHeaderStyles, title})}
  </template>
  </main-view>`;

Keen observers may notice an opportunity here. We don't have to always use the styles and title prescribed in Header.ts, but use the call to appHeaderTemplate as a means to override styling or set the title dynamically. We'll do the latter in a later section of this workshop.

Supporting Declarative Shadow DOM in PostView

Before we code the Express middleware needed to server-side render these Declarative Shadow DOM templates, we have some housekeeping to do. The second view component defined in packages/client/src/view/post/index.ts also needs a function named template exported from the file. This component is responsible for displaying a single blog post.

Notice a pattern that is forming? Patterns are very helpful when server-side rendering components. If each component reliably exports the same named template, we can ensure the middleware reliably interprets each Declarative Shadow DOM template. Standardization is helpful here.

Open packages/client/src/view/post/index.ts. Cut and copy the template declared imperatively into a function named shadowTemplate, just like you did in the last two components.

const shadowTemplate = () => html`<style>
    ${styles}
  </style>
  <div class="post-container">
    <app-header></app-header>
    <div class="post-content">
      <h2>Author: </h2>
      <footer>
        <a href="/">👈 Back</a>
      </footer>
    </div>
  </div>`;

Call shadowTemplate() in the constructor of PostView.

template.innerHTML = shadowTemplate();

Declare a new function named template and make sure to encapsulate the shadowTemplate with Declarative Shadow DOM.

const template = () => html`<post-view>
  <template shadowrootmode="open">
    ${shadowTemplate()}
  </template>
</post-view>`;

Export template from packages/client/src/view/post/index.ts.

export { PostView, template };

We'll return to this file later, but for now that should be enough to display a bit of text and a link that navigates the user back to MainView.

Onto the middleware...

Express Middleware

@lit-labs/ssr is an npm package distributed by Lit at Google for server-side rendering Lit templates and components. The package is for use in the context of Node.js and can be used in conjunction with Express middleware. Express is a popular HTTP server for Node.js that is largely based on middleware.

Express middleware are functions that intercept HTTP requests and handle HTTP responses. We'll code an Express middleware that handles requests to http://localhost:4444 and http://localhost:4444/post/:slug Whenever a user lists these two routes, Express will calls a custom middleware function already exported as ssr. The algorithm in that function is what you'll be working on.

The middleware you'll be working with is imported and set on two routes in packages/server/src/index.ts.

import ssr from "./middleware/ssr.js";
...
app.get("/", ssr);
app.get("/post/:slug", ssr);

When the user visits http://localhost:4444/ or http://localhost:4444/post/, the middleware is activated. The notation :slug in the second route for the post view denotes the middleware should expect a path segment named "slug" that will now appear on the request params passed into the middleware function.

If you've never coded Node.js, have no fear. We'll go step-by-step to code the Express middleware.

Open packages/server/src/middleware/ssr.ts to get started coding the middleware. Browse the file to acquaint yourself with the available import and declared const.

Notice how different files from the monorepo are injected into the HTML declared in renderApp. Relative paths are used here with the readFileSync to find the path of the global styles file named style.css and then minify the styles (for increased performance). The minification could be turned off by dynamically setting the minifyCSS with the env variable, which is used to determine if the code is running in "development" or "production" environments.

const stylePath = (filename) =>
  resolve(`${process.cwd()}../../style/${filename}`);
const readStyleFile = (filename) =>
  readFileSync(stylePath(filename)).toString();

const styles = await minify(readStyleFile('style.css'), {
  minifyCSS: env === "production",
  removeComments: true,
  collapseWhitespace: true,
});

In the above example, the global styles of the blog are read from a file in the monorepo and then minified depending on the environment.

The middleware function is found at the bottom of the file. Currently, the middleware calls renderApp() and responds to the HTTP request (req) by calling res.status(200), which is the HTTP response code for "success" and then .send() with the ES2015 template string returned from renderApp.

export default async (req, res) => {
  const ssrResult = renderApp();
  res.status(200).send(ssrResult);
};

Select the boilerplate shown in the below image and remove the <script>.

Insert a new string in the template to reveal a change in the browser window. In the below example, we inject "Hello World" into the template. This will be temporary, as the renderApp function will become dynamic soon.

function renderApp() {
  return `<!DOCTYPE html>
  <html lang="en">
    <head>
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <meta name="description" content="Web Components Blog">
        <style rel="stylesheet" type="text/css">${styles}</style>
    </head>
    <body> 
      <div id="root">${'Hello World'}</div>
  </body></html>`;
}

Handling Routes in Middleware

Since this middleware will handle multiple routes in Express, we need a way to detect if a route should be served. When a route isn't available, the middleware should throw a 404 Not Found error. HTML should only be served if a route is declared.

In packages/client/src/routes.js each route is declared in an Array. You can import this Array directly from the file, ensuring to use the file extension ".js". This is how imports are handled with ES2015 modules in Node.js, which mirrors the way browsers expect imports to be declared. Eventhough we're coding in TypeScript files, ".js" is still used in the path.

import { routes } from '../../../client/src/routes.js';

Each Route is typed as follows. You can reference this type to code an algorithm that returns the Route based on the HTTP request. You may also rely on your IDE intellisense.

export type Route = {
  component: string;
  path?: string;
  pathMatch?: RegExp;
  tag: string;
  template: (data?: any) => string;
  title?: string;
  params?: any;
};

In the middleware function, write an algorithm that handles two different use cases:

If the route declares a "path", match the route exactly to the current originalUrl on the HTTP request.
If the your declares a "pathMatch", which is a way to match routes by RegExp, call test on the RegExp to determine if the regular expression matches the originalUrl on the HTTP request.

When you are finished, log the matched route. It should log the Route in your Terminal. An edge case should be accounted for when the user visits http://localhost:4444 instead of http://localhost:4444/ and the path is declared as /, there should still be a match.

export default async (req, res) => {
  let route = routes.find((r) => {
    // handle base url
    if (
      (r.path === '/' && req.originalUrl == '') ||
      (r.path && r.path === req.originalUrl)
    ) {
      return r;
    }
    // handle other routes
    if (r.pathMatch?.test(req.originalUrl)) {
      return r;
    }
  });

  console.log(route);
  ...

If there isn't a match, Array.prototype.find will return undefined, so account for this by redirecting the user to a "404" route. We won't actually work on this route now, but for bonus points you could later server-side render a 404 page.

  if (route === undefined) {
    res.redirect(301, '/404');
    return;
  }

Next, add the current HTTP request params to the Route. This will be necessary for the single post view which has a single param :slug which can now be accessed via route.params.slug.

  route = {
    ...route,
    params: req.params,
  };

Now that you have a matched route, you should have access to the route's template stored on the Route, but we first have to "build" the route in development like it were built in production. When the routes are built in the client package, each route is deployed to packages/client/dist as a separate JavaScript bundle. We can simulate this deployment in the development environment by using esbuild programmatically to build the JavaScript bundle that matches each route.

First, define a new function named clientPath that returns the path to the either the view's source in the src directory or the bundle in the dist directory of the client package. We'll need both paths because during development we'll build each view from the src directory to dist. Return a String using resolve and process.cwd() to find the path of the custom element bundle.

To be clear, the function needs to return either of these paths:

packages/client/src/view/main/index.js
packages/client/dist/view/main/index.js
packages/client/src/view/post/index.js
packages/client/dist/view/post/index.js

The first argument of the function should denote the directory, while the second should provide a means to identify the file (stored as route.component).

const clientPath = (directory: 'src' | 'dist', route: any) => {
  return resolve(
`${process.cwd()}../../client/${directory}/view/${route.component}/index.js`
  );
};

Use the new clientPath function in the context of the middleware to build the JavaScript for the route, calling esbuild.build with the path to the source file mapped to entryPoints and the outfile mapped to the path to the distributed bundle in the dist directory. Every time the middleware gets called and matches a route, the appropriate bundle will be built. This will simulate how the bundles are distributed for production and utilize esbuild for development which should be fast and efficient.

 if (env === 'development') {
    await esbuild.build({
      entryPoints: [clientPath('src', route)],
      outfile: clientPath('dist', route),
      format: 'esm',
      minify: env !== 'development',
      bundle: true,
    });
}

Rendering the Template

To render the Declarative Shadow DOM template exported from each bundle, we need to dynamically import the JavaScript from the bundle. Reuse the clientPath function, this time in the context of a dynamic import statement to set a new const named module. This will give us access to whatever is exported from the view's JavaScript bundle.

  const module = await import(clientPath('dist', route));

Declare another const named compiledTemplate that calls template function exported from module. template returns the Declarative Shadow DOM template we defined earlier in packages/client/src/view/main/index.ts.

  const compiledTemplate = module.template();

Before we pass the String returned by the template function to another function named render exported from @lit-labs/ssr, we need to sanitize the HTML. render will ultimately be the function that parses and streams the Declarative Shadow DOM template for the HTML response. We could also provide a custom sanitization algorithm. Whatever the outcome we need to pass the template through the unsafeHTML function exported from @lit-labs/ssr. You could think of this function like dangerouslySetInnerHTML from React. The render function exported from @lit-labs/ssr expects any HTML template to be sanitized through unsafeHTML prior to calling render.

Make a new function named sanitizeTemplate that should be marked async because unsafeHTML itself is asynchronous. html in this example is exported from the lit package, it's the function used by the Lit library for handling HTML templates. It's very similar in nature to the html we tagged the template literals with earlier in the client code, but works well within the Lit ecosystem.

export const sanitizeTemplate = async (template) => {
  return await html`${unsafeHTML(template)}`;
};

In the middleware function, make a new const and set it to a call to sanitizeTemplate, passing in the compiledTemplate. Finally, pass template to renderApp.

const template = await sanitizeTemplate(compiledTemplate);
const ssrResult = renderApp(template);

Add a first argument to renderApp appropriately named template. Replace the "Hello World" from earlier with a call to render, passing in the template. render is the function exported from @lit-labs/ssr that is responsible for rendering the Declarative Shadow DOM template for the entire view. There is some logic render for handling components built with Lit, but it can also accept vanilla Declarative Shadow DOM templates.

function renderApp(template) {
  return `<!DOCTYPE html>
  <html lang="en">
    <head>
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <meta name="description" content="Web Components Blog">
        <style rel="stylesheet" type="text/css">${styles}</style>
    </head>
    <body> 
      <div id="root">${render(template)}</div>
  </body></html>`;
}

If you use intellisense to reveal more about render, you'll learn render accepts any template that can be parsed my lit-html, another package in the Lit ecosystem. You'll also find out the function returns a "string iterator".

By now you'll notice the template doesn't render correctly. Instead [object Generator] is printed in the browser. What is going on here? The hint comes from what render returns: a "string iterator". Generator was introduced in ES2015 and it's an often under appreciated, yet powerful aspect of the JavaScript language. Lit is using Generators to support streaming the value over the request/response lifecycle of a HTTP request.

Handling The Generator

An easy way to support the output of render is to convert renderApp to a Generator function. That is rather easy. Generator can be defined using the function* syntax, which defines a function that returns a Generator. Calling a Generator does not immediately execute the algorithm defined in the function* body. You must define yield expressions, that specify a value returned by the iterator. Generator are just one kind of Iterator. yield* delegates something to another Generator, in our example render.

Convert renderApp to a Generator function, breaking up the HTML document into logical yield or yield* expressions, like in the below example.

function* renderApp(template) {
  yield `<!DOCTYPE html>
  <html lang="en">
    <head>
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <meta name="description" content="Web Components Blog">
        <style rel="stylesheet" type="text/css">${styles}</style>
    </head>
    <body> 
      <div id="root">`;
  yield* render(template);
  yield `</div>
  </body></html>`;
}

Once you're complete, the output in the browser window should change. Rather anticlimactic, huh?

There is a hint for the reason this is happening in the documentation for render. That function "streams" the result and we haven't yet handled the stream. Before we do, update the bottom of the middleware function, being sure to await the result of renderApp, which is now asynchronous due to being a Generator function.

  const ssrResult = await renderApp(template);

Streaming the Result

Streams in Node.js can be expressed as Buffer. Ultimately what we need to do is convert the stream to a String that can be sent to the client in the HTTP response as an HTML document. Buffer can be converted to a String by calling toString, which also accepts utf-8 formatting as an argument. UTF-8 is necessary because it is defined as the default character encoding for HTML.

Streams are asynchronous. We can use a for combined with an await to push each "chunk" of the stream to a Buffer, then combine all the Buffer using Buffer.concat and convert the result to a UTF-8 encoded String. Make a new function named streamToString that does that, giving it an argument named stream. streamToString return the String.

async function streamToString(stream) {
  const chunks = [];
  for await (let chunk of stream) {
    chunks.push(Buffer.from(chunk));
  }
  return Buffer.concat(chunks).toString('utf-8');
}

The above seems like it would just accept our stream returned from renderApp, but we first should pass the stream through Readable, a utility in Node.js that allows us to read the stream. Make another async function named renderStream.

async function renderStream(stream) {
  return await streamToString(Readable.from(stream));
}

Finally, in the middleware function make another variable named stream and call renderStream, passing in the ssrResult. Then, call res.send with the stream.

 let stream = await renderStream(ssrResult);
 res.status(200).send(stream);

Voilà! The title defined in the Declarative Shadow DOM template is now visible in the browser.

If you inspect the HTML you'll notice artifacts left over from the render function. These comments are left behind by Lit and they seem useful possibly for parsing library specific code. Since we just used browser spec, our code may not use them really.

*Declarative Shadow DOM template has successfully been server-side rendered. *

The hard part is over. We'll return to the middleware to enhance parts of it later, but a large majority of the work is done. The middleware you just coded can now be used to render Declarative Shadow DOM templates.

Let's add some content to the view. Next, you'll update the Card custom element to export a Declarative Shadow DOM template, then import the template into the MainView custom element. Then, you'll populate the view with cards that display metadata about each blog post.

Updating Card with Declarative Shadow DOM

In this section, you'll learn how to handle HTML template slots with Delcarative Shadow DOM templates after working on the most complex template yet, a reusable card that only accept content through HTML template slots.

To update the Card open packages/client/src/component/card/Card.ts. Just like in previous examples, cut and paste the template that is declared imperatively in the constructor to a template string returned by a new function named shadowTemplate. Add a single argument to this function that allows you to pass in the styling for the template. Notice how the header, content, footer layout of each Card accepts a named HTML template slot.

const shadowTemplate = ({ styles }) => html`
  <style>
    ${styles}
  </style>
  <header>
    <slot name="header"></slot>
  </header>
  <section>
    <slot name="content"></slot>
  </section>
  <footer>
    <slot name="footer"></slot>
  </footer>
`;

Set the innerHTML in the custom element constructor to the new shadowTemplate function.

template.innerHTML = shadowTemplate({ styles });

Link in the example of Header, Card is a leaf-node, so we can reuse the shadowTemplate wholesale within the Declarative Shadow DOM template. Make a new function named template that passes in a single argument and encapsulate the shadowTemplate with the proper syntax.

const template = ({ styles }) => html`<app-card>
  <template shadowrootmode="open"> 
  ${shadowTemplate({ styles })}
  </template>
</app-card>`;

Cards should display a thumbnail, the blog post title, a short description, and link. Declare new properties on the first argument of the template function that account for content, headline, link, and thumbnail, in addition to styles.

We can still project content into the slots in a Declarative Shadow DOM template. Shadow DOM is contained within the shadowTemplate function. Whatever elements are placed outside of the <template> are considered Light DOM and can be projected through the slots using the slot attribute.

For each of the properties on the model, make a new element. A new <img> tag with a slot attribute set to header will project into the custom element's <header>. Set the src attribute to an interpolated ${thumbnail} and the alt to an interpolated ${content}, to describe the image for screen readers. Additionally define a <h2> and <p>. If any part of the template becomes too complex, you can wrap the entire template partial in string interpolation. This is the case with setting the href attribute on the <a> tag, which displays a "Read Post" link for the user.

const template = ({
  content,
  headline,
  link,
  thumbnail,
  styles,
}) => html`<app-card>
  <template shadowrootmode="open"> ${shadowTemplate({ styles })} </template>
  <img slot="header" src="${thumbnail}" alt="${content}" />
  <h2 slot="header">${headline}</h2>
  <p slot="content">${content}</p>
  ${html`<a href="/post/${link}" slot="footer">Read Post</a>`}
</app-card>`;

Since each element utilizes a named HTML template slot that matches a slot found in Shadow DOM for this custom element, each element will be project into Shadow DOM. If two elements share a named slot, they will be projected in the order they are defined.

Finally, export styles and template from the file.

export { styles, template, AppCard };

We'll switch our attention now to packages/client/src/view/main/index.ts where we need to update the Declarative Shadow DOM template to accept the new template exported from Card.ts.

Fetching the Data Model

Views in modern websites are rarely static. Usually a view must be populated with content from a database. In our project, data is stored in markdown files for each blog post. Either way, a REST API could be used to fetch data from an endpoint and populate the view with content. In this next section, we'll develop a pattern for fetching data from a REST API and integrating the resulting JSON with the middleware you coded in a pervious section.

You'll inject the data from two endpoints into the MainView Declarative Shadow DOM template. JSON returned from 'http://localhost:4444/api/meta can be used to render AppHeader, while the Array of Post returned from http://localhost:4444/api/posts will be used to render a list of AppCard.

Open packages/client/src/view/main/index.ts to begin coding this section.

Just like we did with template, we are standardizing a new pattern. This time for declaring asynchronous API calls that fetch data for the purpose of injecting that data into each Declarative Shadow DOM template. Essentially, we're talking about the "model" for the "view", so we'll call the new function "fetchModel".

This function should return a Promise. With TypeScript, we can strictly type define the Promise and match that definition with the first argument of template. Eventually this data model will get passed to template, enabling us to hydrate the view with content.

function fetchModel(): Promise<DataModel> {}

const template = (data: DataModel) => string;

This may seem disconnected at first, because nowhere in index.ts do these two function work together. In the Express middleware we have access to exports from the bundle, so if we export fetchModel, we can also call the function in the context of the middleware and pass the result to template, which expects the same DataModel. This is why we need a standardized pattern!

An implementation of fetchModel for MainView would be as follows. Use Promise.all to call each endpoint with fetch, then map each response to the JSON returned from the endpoints, and finally call then again to map the resulting JSON to the schema expected in DataModel. The JSON response from http://localhost:4444/api/meta will be used to populate AppHeader, while http://localhost:4444/api/posts will be used to populate content for a list of AppCard.

function fetchModel(): Promise<DataModel> {
  return Promise.all([
    fetch('http://localhost:4444/api/meta'),
    fetch('http://localhost:4444/api/posts'),
  ])
    .then((responses) => Promise.all(responses.map((res) => res.json())))
    .then((jsonResponses) => {
      const meta = jsonResponses[0];
      const posts = jsonResponses[1].posts;
      return {
        meta,
        posts,
      };
    });
}

Update template with a new argument named data and type define it as DataModel. Update the appHeaderTemplate title with the title accessed by data.meta.title.

const template = (data: DataModel) => html`<main-view>
  <template shadowrootmode="open">
    ${appHeaderTemplate({ styles: appHeaderStyles, title: data.meta.title })}
  </template>
  </main-view>`;

We need to integrate the cards that display a preview of each blog post. Import AppCard, styles and template from Card.js, being sure to rename imports where appropriate so they don't clash with any local variables.

import {
  styles as appCardStyles,
  template as appCardTemplate,
  AppCard,
} from '../../component/card/Card.js';

Integrate appCardTemplate into the template function, mapping the Array found at data.posts to the model necessary to display each Card. You need to convert the Array to a String, so call join with an empty String as the argument to convert the Array to a String. Optionally, use the utility named joinTemplates imported into the file instead of calling join directly. Also inject the styles for each Card here.

const template = (data: DataModel) => html`<main-view>
  <template shadowrootmode="open">
    <style>
      ${styles}
    </style>
    <div class="post-container">
    ${appHeaderTemplate({ styles: appHeaderStyles, title: data.meta.title })}
    ${data.posts
      .map(
        (post) =>
          `${appCardTemplate({
            styles: appCardStyles,
            headline: post.title,
            content: post.excerpt,
            thumbnail: post.thumbnail,
            link: post.slug,
          })}`
      )
      .join('')}
     </div>
  </template>
  </main-view>`;

Export all of the following from index.ts.

export { template, fetchModel, AppCard, AppHeader, MainView };

Handling fetchModel in Middleware

To integrate the fetchModel exported from the bundle into the middleware, open packages/server/src/middleware/ssr.ts.

At the top of the middleware function declare a new variable named fetchedData. Declare a let here because sometimes fetchedData may be populated with a function, for static routes, possibly not.

export default async (req, res) => {
  let fetchedData;

After the line where we import the bundle, check if the ES2015 module exports fetchModel with a conditional expression. If truthy, set fetchedData to the result of module.fetchModel() using an await because fetchModel returns a Promise. We don't necessarily want to make fetchModel required for any hypothetical static layouts.

Anticipating the next route that displays a single post, pass in the route to fetchModel. Our existing implementation will effectively ignore this argument, but we need information on the route.params in the next example. Finally, pass fetchedData into the call for module.template.

  const module = await import(clientPath('dist', route));

  if (module.fetchModel) {
    fetchedData = await module.fetchModel(route);
  }

  const compiledTemplate = module.template(fetchedData);

You should now be able to view a server-side rendered header and list of cards at http://localhost:4444. This view is rendered entirely server-side. Network requests are made server-side, the Declarative Shadow DOM template is constructed and then iterated upon by @lit-labs/ssr render Generator. Next, you'll take what you've learned and render a single post view using the same methods.

Rendering A Single Post

If you click on any "Read More" links you'll be greeted with a rather unimpressive layout. A blank "Author:" field and a hand pointing "Back" should greet you. This is the result of the boilerplate we worked on earlier.

Upon navigation to this view, you should notice the Terminal update with a different Route if you still have the console.log enabled.

In this section we're going to populate the single post view with the content of a blog post. Each blog post is written in markdown and stored in the packages/server/data/posts directory. Each post can be accessed via a REST API endpoint at http://localhost:4444/api/post/:slug. If there is a post with a matching "slug", the endpoint returns the blog post in JSON, with the markdown found on the JSON response.

Open packages/client/src/view/post/index.ts to begin coding against the single post view.

import {
  styles as appHeaderStyles,
  template as appHeaderTemplate,
  AppHeader,
} from '../../component/header/Header.js';

  const template = (data: DataModel) => html`<post-view>
  <template shadowrootmode="open">
    <style>
      ${styles}
    </style>
    <div class="post-container">
      ${appHeaderTemplate({ styles: appHeaderStyles, title: data.post.title })}
      <div class="post-content">
        <h2>Author: ${data.post.author}</h2>
        ${data.html}
        <footer>
          <a href="/">👈 Back</a>
        </footer>
      </div>
    </div>
  </template>
</post-view>`;

We can take the fetchModel function from the main view and modify it for the single post view. Copy and paste the function from main/index.ts to post/index.ts, modifying it to accept a new argument. Remember when we passed the route to fetchModel in the middleware? This is why. We need the slug property found on route.params to make the request to http://localhost:4444/api/post/:slug. Modify the fetchModel function until you get a working example, like below.

After we receive the markdown, convert the markdown into useable HTML. The GitHub API is provided in this file to help with this purpose. The blog posts contain code snippets and Octokit is a convenient utility for parsing the markdown and converting it into HTML. Set a new const named request with the response from both local API endpoints and then await another HTTP request to the OctoKit API by calling octokit.request. We want to make a POST request to the /markdown endpoint, passing in the request.post.content to the text property on the body of the request, while making sure to specific the API version in the headers.


function fetchModel({ params }): Promise<any> {
  const res = async () => {
    const request = await Promise.all([
      fetch('http://localhost:4444/api/meta'),
      fetch(`http://localhost:4444/api/post/${params['slug']}`),
    ])
      .then((responses) => Promise.all(responses.map((res) => res.json())))
      .then((jsonResponses) => {
        return {
          meta: jsonResponses[0],
          post: jsonResponses[1].post,
        };
      });

    const postContentTemplate = await octokit.request('POST /markdown', {
      text: request.post.content,
      headers: {
        'X-GitHub-Api-Version': '2022-11-28',
      },
    });
    return {
      ...request,
      html: postContentTemplate.data,
    };
  };
  return res();
}

postContentTemplate will return the converted Markdown into HTML via the data property. Return that in the fetchModel function, along with the meta and post on a new property named html. When you are finished, export all the relevant parts for the middleware.

export { template, fetchModel, AppHeader, PostView };

If you refresh or navigate back to /, then click "Read More" on any card, you now should be able to view a single blog post. You just completed your second server-side rendered view with Declarative Shadow DOM! This is the last view of the workshop. Remaining sections will cover additional content for handling SEO, hydration, and more.

Handling Metadata for SEO

In this particular scenario, a lot can be gleaned from the blog post content for SEO. Each markdown file has a header which contains metadata that could be used for SEO purposes.

---
title: "Form-associated custom elements FTW!"
slug: form-associated-custom-elements-ftw
thumbnail: /assets/form-associated.jpg
author: Myself
excerpt: Form-associated custom elements is a web specification that allows engineers to code custom form controls that report value and validity to `HTMLFormElement`...
---

This header gets transformed by the http://localhost:4444/api/post API endpoint into JSON using the matter package. You can review for yourself in packages/server/src/route/post.ts.

You could expand upon this content, delivering relevant metadata for JSON-LD or setting <meta> tags in the HTML document. For the purposes of this workshop, we'll only set one aspect of the document.head, the page <title>, but you can extrapolate on this further and modify renderApp to project specifications.

Open packages/server/src/middleware/ssr.ts to get started for coding for this section and navigate to the middleware function. Some point before the multiple await, we can pass relevant SEO metadata to the Route. If we wanted to override the route.title with the title of each blog post, we could do that by setting the property with the value returned from fetchedData.meta.title.

  route.title = route.title ? route.title : fetchedData.meta.title;

Pass route into the renderApp function.

  const ssrResult = await renderApp(template, route);

Set the <title> tag with the route.title.

function* renderApp(template, route) {
  yield `<!DOCTYPE html>
  <html lang="en">
    <head>
        <title>${route.title}</title>
   ...

Wherever you derive SEO metadata could vary per project, but in our case we can store this metadata on each markdown file. Since we have repeatable patterns like Route and fetchModel, we can reliably deliver metadata to each HTML document server-side rendered in the middleware.

Hydration

So far all the coding you've done has been completely server-side. Any JavaScript that ran happened on the server and each Declarative Shadow DOM template was parsed by the browser. If there's any other JavaScript in the custom element that needs to run in the browser, like binding a callback to a click listener, that isn't happening yet. We need to hydrate the custom elements.

There's an easy and performant fix. We could host the bundle for each route locally and add a script tag to the HTML document that makes the request, but a more performant solution would be to inline the JavaScript, eliminating the network request entirely.

readFileSync is a Node.js utility that allows us to read the bundle from a file and convert the content to a String. Set the String to a new const named script.

  const module = await import(clientPath('dist', route));
  const script = await readFileSync(clientPath('dist', route)).toString();

Pass the script to the renderApp function in a third argument.

const ssrResult = await renderApp(template, route, script);

Add a <script> tag to the end of the HTML document, setting the content with the String named script.

function* renderApp(route, template, script) {
  yield `<!DOCTYPE html>
  <html lang="en">
    <head>
        <title>${route.title}</title>
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <meta name="description" content="Web Components Blog">
        <style rel="stylesheet" type="text/css">${styles}</style>
    </head>
    <body> 
      <div id="root">`;
  yield* render(template);
  yield `</div>
    <script type="module">${script}</script>
  </body></html>`;
}

We can test hydration is now available in AppHeader. Open
packages/client/src/component/header/Header.ts.

Add an else statement to the conditional already defined in the constructor. This conditional if (!this.shadowRoot) is checking if a shadow root doesn't exist and if truthy, imperatively declares a new shadow root. Since the custom element has a template declared with Declarative Shadow DOM, the content inside the if won't run. We can use an else to run additional logic, client-side only. Modify the <h1> that already exists (because it was server-side rendered) by appending Hydrated to the element innerText.

class AppHeader extends HTMLElement {
  constructor() {
    super();
    if (!this.shadowRoot) {
      const shadowRoot = this.attachShadow({ mode: 'open' });
      const template = document.createElement('template');
      template.innerHTML = shadowTemplate({ styles, title });
      shadowRoot.appendChild(template.content.cloneNode(true));
    } else {
      const title = this.shadowRoot.querySelector('h1').innerText;
      this.shadowRoot.querySelector('h1').innerText = `${title} Hydrated`;
    }
  }
}

Remove the else once your test is complete. This section demoed how to hydrate custom elements client-side, but how does the server understand any of this JavaScript? Some of you maybe wondering how did the server interpret class AppHeader extends HTMLElement when HTMLElement doesn't exist in Node.js. This final section is an explainer of how this works with @lit-labs/ssr. Coding is complete. Congratulations! You've finished the workshop.

Shimming Browser Spec

Lit shims browser specifications it needs to run on the server via a function exported from @lit-labs/ssr/lib/dom-shim.js named installWindowOnGlobal. If all your project relies on is autonomous custom elements extended from HTMLElement you can get by just with calling this function prior to anything else in Node.js.

A custom implementation of the shim is found at packages/shim/src/index.ts. I used this custom shim for the chapters in the book Fullstack Web Components because the views we server-side render in the book contain Customized built-in elements, which are not shimmed by default. Lit was thoughtful enough to allow engineers to extend the shim with other mocks necessary to server-side render browser specifications. Use this file as an example of how you could shim browser spec in your server-side rendered project.

installShimOnGlobal is exported from this package in the monorepo and imported into packages/server/src/index.ts where it's called before any other code.

import { installShimOnGlobal } from "../../shim/dist/index.js";

installShimOnGlobal();

It's necessary to shim browser specifications before any other code runs in Node.js. That's why the shim is executed first.

Conclusion

In this workshop, you server-side rendered a blog using Declarative Shadow DOM and the @lit-labs/ssr package with Express middleware. You took autonomous custom elements that declared a shadow root imperatively and made reusable templates that also support Declarative Shadow DOM. Declarative Shadow DOM is a standard that allows web developers to encapsulate styling and template with a special HTML template, declaratively. The "special" part of the HTML template is the element must use the shadowrootmode attribute.

You learned how to use the render Generator exported from @lit-labs/ssr to server-side render template partials in a HTML document. We only rendered one template, but you could take what you learned to render multiple template partials, depending on the needs of the project.

The usage of a static configuration for each Route, exporting template and fetchModel from each view bundle were opinions. This is just one way of working. The main takeaway is for server-side rendering, you should standardize portions of the system to streamline development and ease with integration.

We didn't cover the entire build system in this workshop. Under the hood, nodemon was watching for changes while esbuild is responsible for building for production. If you build for production, you will find the output is 100% minified and scores 100% for performance in Lighthouse. Someone could streamline development further with Vite, the main benefit one would gain is hot module reloading.

We also didn't cover how to use @lit-labs/ssr with LitElement and that was on purpose. Lit is a wonderful library. Use it at your discretion. I find it much more interesting that Lit, because it is built from browser specifications, can operate with browser spec like Declarative Shadow DOM without coding directly with Lit. I hope you find this interesting as well. There is another aspect to this I would like to highlight.

When I started in web development in the 90s, I appreciated the egalitarian nature of HTML and JavaScript. Anyone could code a website, like anyone could read and write a book. Over time, front end web development has gotten way too complicated. Frameworks and libraries that sought to simplify developer experience did so at the cost of user experience, while also making the barrier to entry far greater for anyone wanting to learn how to code a site. I demonstrated how to server-side render a website using only browser standards and leveraging a few tools from Lit. Anyone can learn how to code Declarative Shadow DOM, just like decades ago anyone could learn how to code HTML.

I hope you found it easy to accomplish server-side rendering with Declarative Shadow DOM and can't wait to see what you build. Comment below with a link so we can see those 100% performance scores in Lighthouse.

If you liked this tutorial, you will find many more like it in my book about Web Components.

Fullstack Web Components

Are you looking to code Web Components now, but don't know where to get started? I wrote a book titled Fullstack Web Components, a hands-on guide to coding UI libraries and web applications with custom elements. In Fullstack Web Components, you'll...

Code several components using autonomous, customized built-in, and form-associated custom elements, Shadow DOM, HTML templates, CSS variables, and Declarative Shadow DOM
Develop a micro-library with TypeScript decorators that streamlines UI component development
Learn best practices for maintaining a UI library of Web Components with Storybook
Code an application using Web Components and TypeScript

Fullstack Web Components is available now at newline.co

Blog