Introduction

In modern web applications, enhancing user experience demands high performance and smooth interactions. However, executing complex calculations or data processing on the UI thread can lead to delayed rendering and reduced responsiveness of the app. To address these issues, I have conducted the following experiments:

• Utilizing Web Workers to offload heavy computational tasks to a background thread, thereby reducing the load on the UI thread.
• Leveraging RxJS to manage data flows in a concise and efficient manner, improving code readability.

This article will explore how to achieve high performance while efficiently managing complex data processing by utilizing Web Workers and RxJS, explained through specific demos and code snippets.

DEMO

Tested on MacBook Pro & Chrome. I created a demo that processes and visualizes camera and mouse input.

DISTRIBUTION WORKER
The repository is also available for viewing if you're interested.

Background

The demo processes images received from the camera, coloring denser areas in red, lighter areas in blue, and medium intensity areas in green. Areas with more movement have been given a higher pixel height.

Indicator

This is an indicator that fills up by 10% every 0.8 seconds while the mouse is clicked.

Mouse Movement History

It aggregates mouse movements once every second and displays the number of occurrences and the amount of movement. If there was no movement, it displays "did not happen...".

Notifications

Displays events that occur within the app. Notifications are categorized into four types: Info, Succeed, Failed, and Warning, each changing the background color and icon accordingly.

Switching the Execution Context of Computational Tasks

Switches whether computational tasks are performed on the Main Thread or on a Web Worker.

Structure

In reactive web applications, the sharing of state is often discussed. However, there are some challenges:

1. The value at the time of writing differs from the value at the time of reading.
2. You either keep values that should only be read once forever or need to include a process to delete them.

Therefore, I thought that sharing "events" rather than values suits our requirements better for this case.

I will introduce the detailed structure later, but I have established a rule that elements in src/features/* should not depend on each other. The reasons for choosing the libraries used are as follows:

SolidJS

I like JSX, so I compared it with React. The main focus of the project is canvas operations. With React, you'd need to prevent re-rendering with hooks like useMemo and useCallback, but SolidJS does not require such measures, which I found appealing. Since canvas processing itself is full of side effects, SolidJS, which performs only the minimum necessary rendering, seemed more suited to our requirements.

Three.js

Although part of the reason was that I could not find any other compelling libraries, the candidates for 3D libraries were Three.js and Babylon.js. For this verification, since the goal was to be able to write out to WebGL, I chose Three.js, with which I had some previous experience.

RxJS

I also looked into Bacon.js and Most.js, but RxJS was the most active and had the best documentation. The ability to easily write and understand filter processes through pipes made RxJS the chosen library for this project.

Project Structure

src/app.tsx

This is the entrypoint. Usually, I write the process for rendering to the DOM in main.tsx and the process that combines various components in app.tsx like this:

// main.tsx
import { App } from './app'
render(<App />, document.getElementById('root')!)

// app.tsx
export function App() {
  return (
    <Layout>
      <A />
      <B />
      <C />
    </Layout>
  )
}

To avoid confusion by splitting the code too much, I combined these codes into src/app.tsx.

src/$/*

This folder manages streams with RxJS. Observables are suffixed with $. Functions that publish are prefixed with emit.

src/features/*

These are the features used in src/app.tsx. By emitting to and subscribing from streams, the sibling elements within features are designed to be independent of each other.

src/lib/*, src/types/*

These are used throughout the project. Along with the previously mentioned src/$/*, the concept was to create features that rely on the concepts of events and types.

From Processing Camera Input to Writing to Canvas

Here, I will explain the code in detail. If you like, please clone the repo and follow along as you run it.

When the app is opened, a button to access the camera is displayed in the center of the screen.

src/features/interaction/no-signal.tsx

When this button is pressed, accessRequested$ in src/$/camera.ts fires. src/features/camera/watch-access-requested.ts subscribes to this stream and accesses the camera. If camera access is successfully granted, mediaStreamCreated$ in src/$/camera.ts fires.

src/features/camera/watch-media-stream-created.ts subscribes to mediaStreamCreated$, attaches the mediaStream to a video tag, and writes the camera image to canvas 2d. At the timing of requestAnimationFrame, it pulls up the camera image and flows it into captured$ in src/$/camera.ts.

src/features/heat-map/render-heat-map.ts subscribes to captured$ and writes to WebGL via Three.js. During this process, the camera image is processed, and whether this is done in a Web Worker or on the Main Thread is determined by the user's choice.

Subscribes to the calculationProcessLocationUpdated$ event in src/$/interaction.ts. location is of the type CalculationProcessLocation.

export type CalculationProcessLocation = 'WebWorker' | 'MainThread'

By using switchMap, it's possible to stop the stream before switching and start the next stream. map is mainly used in the processing phase. In this case, it's used to process the camera image into vertices and colors. tap is used when causing side effects. This operator does not affect the data flowing through the stream.

calculationProcessLocationUpdated$
  .pipe(
    switchMap((location) =>
      location === 'WebWorker'
        ? captured$.pipe(
            map((capture) => ({
              capture,
              size,
              height: window.innerHeight,
            })),
            tap((payload) => worker.postMessage(payload))
          )
        : captured$.pipe(
            map((capture) => calculate(capture, size, window.innerHeight)),
            tap(([vertices, colors]) => renderer.render(vertices, colors))
          )
    )
  )
  .subscribe()

Using Web Workers with Vite

You can develop with Web Workers in TypeScript without having to edit vite.config.ts. However, the import method is somewhat unique, and it won't work after building unless you do it like this:

// Add ?worker
import Worker from './worker?worker'
const worker = new Worker()

To use the same method of calculation in both the Main Thread and the Web Worker, I extracted the function. As a result, the Worker itself ended up being very simple.

import { WorkerPayload } from '~/types/worker-payload'
import { calculate } from './calculate'

self.addEventListener('message', (event: MessageEvent<WorkerPayload>) => {
  const { capture, size, height } = event.data
  const [vertices, colors] = calculate(capture, size, height)
  self.postMessage(
    {
      vertices: vertices.buffer,
      colors: colors.buffer,
    },
    [vertices.buffer, colors.buffer] as any
  )
})

By passing a reference as the second argument to self.postMessage, you can distribute data quickly.

Displaying Long Mouse Presses on an Indicator

I used apexcharts for the chart. It was a library I had never used before, but it was a good opportunity to try it out. The chart implementation is mostly according to the samples provided. It was similar to recharts and easy to use.

I create a stream for long presses by merging pressed$ and released$ located in src/$/mouse.ts. Please see src/features/mouse-pressed-indicator.ts for details.

export const $ = merge(
  pressed$.pipe(map(() => 5)),
  released$.pipe(map(() => 0))
).pipe(
  switchMap((value) =>
    value === 0
      ? [0]
      : interval(800).pipe(
          startWith(value),
          scan((current) => Math.min(current + 10, 100), value - 5),
          takeUntil(released$)
        )
  )
)

Displaying Long Presses on a Mouse as an Indicator

I used apexcharts for the chart. It was a library I hadn't used before, but it seemed like a good opportunity to try it out. The implementation of the chart is mostly as per the examples provided. It was similar to recharts and easy to use.

I created a stream for long presses by merging pressed$ and released$ in src/$/mouse.ts. Please see src/features/mouse-pressed-indicator/$.ts for reference.

Since the indicator does not move immediately after pressing, I set it to start at 5. Trying to capture long presses can lead to complex code, but using RxJS makes it easy to assemble.

Displaying Mouse Movement History

It displays the mouse movement distance and the number of mousemove events once every second. I created a new stream by processing moved$ found in src/$/mouse.ts.

const initialState: MouseMovement = {
  occurred: 0,
  x: 0,
  y: 0,
  amount: 0,
}

export const $ = moved$.pipe(
  bufferTime(1000),
  map((events) =>
    events.reduce<MouseMovement>(
      (previous, current: MouseMoveEvent) => ({
        occurred: previous.occurred + 1,
        x: Math.abs(previous.x) + Math.abs(current.x),
        y: Math.abs(previous.y) + Math.abs(current.y),
        amount: previous.amount + Math.sqrt(current.x ** 2 + current.y ** 2),
      }),
      initialState
    )
  )
)

In this feature, events are buffered for 1000 milliseconds and then reduced to aggregate the data.

Notifications

Until now, I've been subscribing to events from the mouse, camera, etc., which were independent features, so the scope of subscription was limited. However, this notification feature monitors all notifications across the app. Please see the code below.

export const $: Observable<Notification> = merge(
  camera.accessRequested$.pipe(map(() => info('Camera Access Requested'))),
  camera.permissionUpdated$.pipe(
    filter((status) => status === 'granted'),
    map((_) => succeed('Camera Access Granted'))
  ),
  camera.permissionUpdated$.pipe(
    filter((status) => status === 'denied'),
    map((_) => warning('Camera Access Denied'))
  ),
  camera.unavailable$.pipe(map((cause) => failed('Camera Unavailable', cause))),
  camera.mediaStreamCreated$.pipe(map(() => succeed('MediaStream Created'))),
  camera.captured$.pipe(
    take(1),
    map(() => succeed('Heat Map Rendering Started'))
  ),
  interaction.calculationProcessLocationUpdated$.pipe(
    filter((c) => c === 'MainThread'),
    map((_) => info('Process Location', 'Switched to Main Thread'))
  ),
  interaction.calculationProcessLocationUpdated$.pipe(
    filter((c) => c === 'WebWorker'),
    map((_) => info('Process Location', 'Switched to Web Worker'))
  ),
  background.canvasMounted$.pipe(
    map(() => succeed('Background canvas Created'))
  ),
  mouse.pressed$.pipe(map(() => info('Mouse Pressed'))),
  mouse.released$.pipe(map(() => info('Mouse Released'))),
  window.resized$.pipe(
    throttleTime(1000),
    map(({ width, height }) =>
      info(
        'Window Resized',
        `width: ${width.toLocaleString()}, height: ${height.toLocaleString()}`
      )
    )
  )
)

By doing so, I am processing multiple streams according to my own requirements. This does not affect any existing functionality at all. It's the most fun moment when working with RxJS.

The emitting side just needs to faithfully report what has happened, while the subscribing side can insert operators into the pipe to process the data into a form that is easier for them to handle.

Conclusion

The use of Web Workers and RxJS has significantly improved performance. Specifically, it has reduced the load on the UI thread and contributed to an increase in frame rate. Notably, the frame rate when using Web Workers showed a clear improvement compared to using the Main Thread alone.

• Main Thread: Averages around 30FPS
• With Web Worker: Maintains 60FPS

However, running on the Main Thread continuously also resulted in an improvement up to 60FPS. There might be something akin to caching at play.

Moreover, on Retina displays, the frame rate increased to a maximum of 120FPS. When comparing on Retina displays: using Web Workers and using only the Main Thread recorded 120FPS and 60FPS, respectively.

SolidJS

While the main focus of this project was on Web Workers and RxJS, and I didn't touch much on SolidJS, I find it to be an excellent library. When writing in React, you might write something like {nullable && <div>{nullable}</div>}. In SolidJS, you can declare conditions more declaratively with <Show when={nullable}>{nullable}</Show>, which is similar to Vue's v-if and quite nice. Also, instead of writing xs.map(x => <div key={x}>{x}</div>), you can write <For each={xs}>, which allows for more pleasant writing in JSX. Additionally, there was a way to edit drafts, familiar to those who use immer.

setMovements(
  produce((draft) => {
    draft.unshift(movement)
    draft.length > 50 && draft.pop()
  })
)

PR

VOTE, developed by blue Inc., is a web application where you vote on binary choices. From technical topics to everyday decisions, feel free to play around with a variety of topics.