Fun with speech detection using RxJS and Angular standalone components

Introduction

This is day 20 of Wes Bos's JavaScript 30 challenge where I build a speech detection application using RxJS, Angular standalone components and Web Speech API. Angular not only can call it’s own APIs to render components but it can also interact with Web Speech API to guess what I spoke in English and outputted its confidence level. The API amazed me due to its high accuracy and confidence level as if a real human being was listening to me and understood my accent.

Create a new Angular project

ng generate application day20-speech-detection-standalone

Bootstrap AppComponent

First, I convert AppComponent into standalone component such that I can bootstrap AppComponent and inject providers in main.ts.

// app.component.ts

import { Component } from '@angular/core';
import { Title } from '@angular/platform-browser';
import { SpeechDetectionComponent } from './speech-detection/speech-detection/speech-detection.component';

@Component({
  selector: 'app-root',
  standalone: true,
  imports: [SpeechDetectionComponent],
  template: '<app-speech-detection></app-speech-detection>',
  styles: [
    `
      :host {
        display: block;
      }
    `,
  ],
})
export class AppComponent {
  title = 'Day20 Speech Detection Standalone';

  constructor(titleService: Title) {
    titleService.setTitle(this.title);
  }
}

In Component decorator, I put standalone: true to convert AppComponent into a standalone component.

Instead of importing SpeechDetectionComponent in AppModule, I import SpeechDetectionComponent (that is also a standalone component) in the imports array because the inline template references it. It is because main.ts uses bootstrapApplication to render AppComponent as the root component of the application. When compiler sees <app-speech-detection> in the inline template and AppComponent does not import SpeechDetectionComponent, the application fails to compile.

// main.ts

import { bootstrapApplication } from '@angular/platform-browser';
import { AppComponent } from './app/app.component';

bootstrapApplication(AppComponent).catch((err) => console.error(err));

Next, I can delete AppModule that has no use now.

Declare speech detection component

I declare standalone component, SpeechDetectionComponent, to start the speech detection process. To verify the component is standalone, standalone: true is specified in the Component decorator.

src/app/
├── app.component.spec.ts
├── app.component.ts
└── speech-detection
    ├── helpers
    │   └── speech-detection.helper.ts
    ├── interfaces
    │   └── speech-recognition.interface.ts
    └── speech-detection
        ├── speech-detection.component.spec.ts
        └── speech-detection.component.ts

The application use this component to do everything and the tag is <app-speech-detection>.

// speech-detection.component.ts

import { AsyncPipe, NgFor, NgIf } from '@angular/common';
import { ChangeDetectionStrategy, Component, OnDestroy, OnInit } from '@angular/core';
import {
  createRecognition,
  createRecognitionSubscription,
  createWordListObservable,
} from '../helpers/speech-detection.helper';

@Component({
  selector: 'app-speech-detection',
  standalone: true,
  imports: [AsyncPipe, NgFor, NgIf],
  template: ` <div class="words" contenteditable>
    <ng-container *ngIf="wordList$ | async as wordList">
      <p *ngFor="let word of wordList">{{ word.transcript }}, confidence: {{ word.confidencePercentage }}%</p>
    </ng-container>
  </div>`,
  styles: [`...omitted due to bervity...`],
  changeDetection: ChangeDetectionStrategy.OnPush,
})
export class SpeechDetectionComponent implements OnInit, OnDestroy {
  recognition = createRecognition();
  subscription = createRecognitionSubscription(this.recognition);
  wordList$ = createWordListObservable(this.recognition);

  ngOnInit(): void {
    this.recognition.start();
  }

  ngOnDestroy(): void {
    this.subscription.unsubscribe();
  }
}

SpeechDetectionComponent imports AsyncPipe, NgFor and NgIf in the imports array because the inline template makes use of ngIf, ngFor and async.

Add speech detection helper to define observable and subscription
I moved speech detection observable and subscription to helper file to maintain small component file and good project structure.

// speech-recognition.interface
export interface SpeechRecognitionInfo {
  transcript: string;
  confidence: number;
  isFinal: boolean;
}

export type Transcript = Omit<SpeechRecognitionInfo, 'isFinal' | 'confidence'> & { confidencePercentage: string };
// speech-detection.helper.ts

import { fromEvent, tap, map, filter, scan } from 'rxjs';
import {
  SpeechRecognitionInfo,
  Transcript,
} from '../interfaces/speech-recognition.interface';

declare var webkitSpeechRecognition: any;
declare var SpeechRecognition: any;

export const createRecognition = () => {
  const recognition = new webkitSpeechRecognition() || new SpeechRecognition();
  recognition.interimResults = true;
  recognition.lang = 'en-US';

  return recognition;
};

export const createRecognitionSubscription = (recognition: any) =>
  fromEvent(recognition, 'end')
    .pipe(tap(() => recognition.start()))
    .subscribe();

export const createWordListObservable = (recognition: any) => {
  const percent = 100;
  return fromEvent(recognition, 'result').pipe(
    map((e: any): SpeechRecognitionInfo => {
      const transcript = Array.from(e.results)
        .map((result: any) => result[0].transcript)
        .join('');
      const poopScript = transcript.replace(/poop|poo|shit|dump/gi, '💩');
      const firstResult = e.results[0];

      return {
        transcript: poopScript,
        confidence: firstResult[0].confidence,
        isFinal: firstResult.isFinal,
      };
    }),
    filter(({ isFinal }) => isFinal),
    scan(
      (acc: Transcript[], { transcript, confidence }) =>
        acc.concat({
          transcript,
          confidencePercentage: (confidence * percent).toFixed(2),
        }),
      [],
    ),
  );
};

Explain createRecognition

createRecognition function instantiates a SpeechRecognition service, sets the language to English and interimResult to true.

Explain createRecognitionSubscription

createRecognitionSubscription function restarts speech recognition and subscribes the Observable after the previous speech recognition ends.

fromEvent(recognition, 'end') – Emit value when end event of speech recognition service occurs
tap(() => recognition.start()) – restart speech recognition to recognize the next sentence
subscribe() – subscribe observable

Explain createWordListObservable

createWordListObservable function emits a list of phrases recognized by Web Speech API.

fromEvent(recognition, 'result') – Speech recognition emits a word or phrase
map() – replace foul languages with emoji and return the phrase, confidential level and the isFinal flag
filter(({ isFinal }) => isFinal) – emit value when the final result is ready
scan() – append the phrase and confidence level to the word list

Demystifies RxJS logic in SpeechDetectionComponent

SpeechDetectionComponent class is succinct now after refactoring Observable and Subscription codes to speech-detection.helper.ts. They are extracted to functions in helper file

recognition = createRecognition();
subscription = createRecognitionSubscription(this.recognition);
wordList$ = createWordListObservable(this.recognition);
The component declares this.recognition that is a SpeechRecognition service.

this.wordList$ is an Observable that renders a list of words or phrases.

<ng-container *ngIf="wordList$ | async as wordList">
    <p *ngFor="let word of wordList">
       {{ word.transcript }}, confidence: {{ word.confidencePercentage }}%
    </p>
</ng-container>

this.subscription is a subscription that unsubscribe in ngOnDestroy

ngOnDestroy(): void {
  this.subscription.unsubscribe();
}

This is it and I have built a speech recognition application with RxJS, Angular standalone components and Web Speech API.

Final Thoughts

In this post, I show how to use RxJS, Angular standalone components and web speech API to build speech detection. The application has the following characteristics after using Angular 15’s new features:

The application does not have NgModules and constructor boilerplate codes.
The standalone component is very clean because I moved as many RxJS codes to separate helper file as possible.

This is the end of the blog post and I hope you like the content and continue to follow my learning experience in Angular and other technologies.