How to Detect and Rectify Documents in Angular Web Applications

Dynamsoft Document Normalizer JavaScript SDK features document detection and rectification. It can be easily integrated into web applications with a few lines of code. In this article, we will create an Angular web application to demonstrate how to use the SDK to detect and rectify documents.

Prerequisites

Angular CLI: A command-line interface tool for Angular development.

  npm install -g @angular/cli

Initialize Angular Project and Install Dependencies

In your terminal, run the following command to create a new Angular project.

ng new angular-document-edge-detection

Then, navigate to the project folder and install the dependencies.

Dynamsoft Document Normalizer: A JavaScript SDK for document detection and rectification.

  npm i dynamsoft-document-normalizer

Dynamsoft Camera Enhancer: A JavaScript SDK for camera control, image capture and video streaming.

  npm i dynamsoft-camera-enhancer

Next, open angular.json file to configure the asset path for Dynamsoft Document Normalizer:

"assets": [
  "src/favicon.ico",
  "src/assets",
  {
    "glob": "**/*",
    "input": "./node_modules/dynamsoft-document-normalizer/dist",
    "output": "assets/dynamsoft-document-normalizer"
  }
],

The output path should be the same as the one set in code. We create a dynamsoft.service.ts file to set the license key and the asset path:

import { Injectable, Optional } from '@angular/core';
import { DocumentNormalizer} from 'dynamsoft-document-normalizer';

@Injectable({
  providedIn: 'root'
})
export class DynamsoftService {

  constructor() {
    DocumentNormalizer.license = "DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==";
    DocumentNormalizer.engineResourcePath = "assets/dynamsoft-document-normalizer";
  }
}

Algorithm Parameters for Document Detection and Rectification

Dynamsoft Document Normalizer provides a set of parameters to control the detection and rectification process. You can refer to the online documentation for more details.

Here we create some simple parameter templates with TypeScript multiline strings:

export class Template {
    static binary = `
      {
          "GlobalParameter":{
              "Name":"GP"
          },
          "ImageParameterArray":[
              {
                  "Name":"IP-1",
                  "NormalizerParameterName":"NP-1",
                  "BinarizationModes":[{"Mode":"BM_LOCAL_BLOCK", "ThresholdCompensation":9}],
                  "ScaleDownThreshold":2300
              }
          ],
          "NormalizerParameterArray":[
              {
                  "Name":"NP-1",
                  "ColourMode": "ICM_BINARY" 
              }
          ]
      }
      `;

    static color = `
      {
          "GlobalParameter":{
              "Name":"GP"
          },
          "ImageParameterArray":[
              {
                  "Name":"IP-1",
                  "NormalizerParameterName":"NP-1",
                  "BinarizationModes":[{"Mode":"BM_LOCAL_BLOCK", "ThresholdCompensation":9}],
                  "ScaleDownThreshold":2300
              }
          ],
          "NormalizerParameterArray":[
              {
                  "Name":"NP-1",
                  "ColourMode": "ICM_COLOUR" 
              }
          ]
      }
      `;

    static grayscale = `
      {
          "GlobalParameter":{
              "Name":"GP"
          },
          "ImageParameterArray":[
              {
                  "Name":"IP-1",
                  "NormalizerParameterName":"NP-1",
                  "BinarizationModes":[{"Mode":"BM_LOCAL_BLOCK", "ThresholdCompensation":9}],
                  "ScaleDownThreshold":2300
              }
          ],
          "NormalizerParameterArray":[
              {
                  "Name":"NP-1",
                  "ColourMode": "ICM_GRAYSCALE"
              }
          ]
      }
      `;
}

The only difference between these three templates is the ColourMode parameter. The ColourMode parameter controls the color mode of the output image. It can be set to ICM_BINARY, ICM_COLOUR or ICM_GRAYSCALE.

Angular Components for Detecting and Rectifying Documents

We are going to create two Angular components: file and camera. The file component is responsible for detecting and rectifying documents from image files. The camera component is responsible for detecting and rectifying documents from camera stream.

ng generate component file-detection
ng generate component camera-detection

File Detection Component

The file detection page consists of five parts: a radio button group, a file input button, an image element, a canvas and a div element.

<span id="loading-status" style="font-size:x-large" [hidden]="isLoaded">Loading Library...</span>
<br />

<div class="row">

    <label for="binary"> <input type="radio" name="templates" value="binary" (change)="onRadioChange($event)" />Black &
        White </label>


    <label for="grayscale"><input type="radio" name="templates" value="grayscale" (change)="onRadioChange($event)" />
        Grayscale </label>

    <label for="color"><input type="radio" name="templates" value="color" [checked]="true"
            (change)="onRadioChange($event)" /> Color </label>
</div>

<input type="file" title="file" id="file" accept="image/*" (change)="onChange($event)" />

<div class="container">
    <div id="imageview">
        <img id="image" alt="" />
        <canvas id="overlay"></canvas>
    </div>

    <div id="resultview">
        <canvas id="normalizedImage"></canvas>
    </div>

</div>

In TypeScript, we first inject the DynamsoftService in the constructor and initialize the Dynamsoft Document Normalizer object in the ngOnInit method.

import { Component, OnInit } from '@angular/core';
import { DocumentNormalizer } from 'dynamsoft-document-normalizer';
import { DynamsoftService } from '../dynamsoft.service';
import { OverlayManager } from '../overlay';
import { Template } from '../template';

@Component({
  selector: 'app-file-detection',
  templateUrl: './file-detection.component.html',
  styleUrls: ['./file-detection.component.css']
})
export class FileDetectionComponent implements OnInit {
  isLoaded = false;
  overlay: HTMLCanvasElement | undefined;
  context: CanvasRenderingContext2D | undefined;
  normalizer: DocumentNormalizer | undefined;
  overlayManager: OverlayManager;
  points: any[] = [];
  currentFile: File | undefined;

  constructor(private dynamsoftService: DynamsoftService) {
    this.overlayManager = new OverlayManager();
  }

  ngOnInit(): void {
    this.overlayManager.initOverlay(document.getElementById('overlay') as HTMLCanvasElement);
    (async () => {
      this.normalizer = await DocumentNormalizer.createInstance();
      this.isLoaded = true;
      await this.normalizer.setRuntimeSettings(Template.color);
    })();
  }
}

Then add implementations for the UI elements.

The radio button group is used to switch the parameter template.

onRadioChange(event: Event) {
  if (!this.normalizer) {
    return;
  }

  let target = event.target as HTMLInputElement;
  let template = Template.binary;
  if (target.value === 'binary') {
    template = Template.binary;
  } else if (target.value === 'grayscale') {
    template = Template.grayscale;
  } else if (target.value === 'color') {
    template = Template.color;
  }
  (async () => {
    await this.normalizer!.setRuntimeSettings(template);
    this.normalize(this.currentFile!, this.points);
  })();
}

The file input button is used to select the image file.

onChange(event: Event) {
  const element = event.currentTarget as HTMLInputElement;
  let fileList: FileList | null = element.files;
  if (fileList) {
    let file = fileList.item(0) as any;

  }
}

The image element is used to display the loaded image file.

if (file) {
  this.currentFile = file;
  let fr = new FileReader();
  fr.onload = (event: any) => {
    let image = document.getElementById('image') as HTMLImageElement;
    if (image) {
      image.src = event.target.result;
      const img = new Image();

      img.onload = (event: any) => {

      };
      img.src = event.target.result;
    }
  };
  fr.readAsDataURL(file);
}

As the image is loaded， invoke the detectQuad method to detect the document edges.

this.normalizer.detectQuad(file).then((results: any) => {
  try {
    if (results.length > 0) {

    } 
  } catch (e) {
    alert(e);
  }
});

The canvas is used to display the detected document edges.

try {
  if (results.length > 0) {
    let result = results[0];
      this.points = result['location']['points'];
      this.overlayManager.drawOverlay(
        this.points,
      );
  } 
} catch (e) {
  alert(e);
}

The div element is used to display the rectified document.

this.normalize(file, this.points);

normalize(file: File, points: any) {
  if (this.normalizer) {
    this.normalizer.normalize(file, points).then((result: any) => {
      let image = document.getElementById('normalizedImage') as HTMLCanvasElement;
      if (image) {
        image.width = result.image.width;
        image.height = result.image.height;
        let ctx = image.getContext('2d') as CanvasRenderingContext2D;

        var imgdata = ctx.createImageData(image.width, image.height);
        var imgdatalen = result.image.data.length;
        for(var i=0; i<imgdatalen; i++) {
            imgdata.data[i] = result.image.data[i];
        }
        ctx.putImageData(imgdata, 0, 0);
      }
    });
  }
}

Camera Detection Component

The UI of the camera detection component is similar to the file detection component. The difference is the image element is replaced by the video element.

<div id="document-scanner">
    <span id="loading-status" style="font-size:x-large" [hidden]="isLoaded">Loading Library...</span>
    <br />
    <div class="row">

        <label for="binary"> <input type="radio" name="templates" value="binary" (change)="onRadioChange($event)" />Black &
            White </label>


        <label for="grayscale"><input type="radio" name="templates" value="grayscale" (change)="onRadioChange($event)" />
            Grayscale </label>

        <label for="color"><input type="radio" name="templates" value="color" [checked]="true"
                (change)="onRadioChange($event)" /> Color </label>
    </div>
    <label for="threshold"><input id="thredshold" (change)="updateThresholdCompensation($event)" type="range" min="0" max="10" value="9" step="1">Edge Detection Threshold: <span id="ThresholdCompensationval" style="width: 12px;" [textContent]="9"></span></label>
    <div>
        <label for="videoSource">Video Source: 
        <select id="videoSource" (change)="openCamera()"></select></label>
        <button id="detectButton" (click)="detectDocument()">Start Detection</button>
        <button id="captureButton" (click)="captureDocument()">Capture Document</button>
    </div>

    <div id="videoview">
        <div class="dce-video-container" id="videoContainer"></div>
        <canvas id="overlay"></canvas>
    </div>

    <div class="container">
        <div id="resultview">
            <canvas id="normalizedImage"></canvas>
        </div>
    </div>
</div>

It is known that the getUserMedia method is the only API that can access the camera stream. With the getUserMedia method, we still have lots of work to do in camera programming. To simplify the coding, we use Dynamsoft Camera Enhancer instead. The JavaScript camera SDK encapsulates the getUserMedia method and provides some useful APIs.

Here is the initialization code for the camera detection component.

import { Component, OnInit } from '@angular/core';
import { DocumentNormalizer } from 'dynamsoft-document-normalizer';
import { CameraEnhancer } from 'dynamsoft-camera-enhancer';
import { DynamsoftService } from '../dynamsoft.service';
import { OverlayManager } from '../overlay';
import { Template } from '../template';

@Component({
  selector: 'app-camera-detection',
  templateUrl: './camera-detection.component.html',
  styleUrls: ['./camera-detection.component.css']
})
export class CameraDetectionComponent implements OnInit {
  isLoaded = false;
  overlay: HTMLCanvasElement | undefined;
  context: CanvasRenderingContext2D | undefined;
  normalizer: DocumentNormalizer | undefined;
  overlayManager: OverlayManager;
  currentData: any;
  cameraInfo: any = {};
  videoSelect: HTMLSelectElement | undefined;
  enhancer: CameraEnhancer | undefined;
  isDetecting = false;
  captured: any[] = [];

  constructor(private dynamsoftService: DynamsoftService) {
    this.overlayManager = new OverlayManager();
  }

  ngOnDestroy() {
    this.normalizer?.dispose();
    this.normalizer = undefined;

    this.enhancer?.dispose(true);
    this.enhancer = undefined;
  }

  ngOnInit(): void {
    this.videoSelect = document.querySelector('select#videoSource') as HTMLSelectElement;
    this.overlayManager.initOverlay(document.getElementById('overlay') as HTMLCanvasElement);
    (async () => {
      this.normalizer = await DocumentNormalizer.createInstance();
      this.enhancer = await CameraEnhancer.createInstance();
      this.enhancer.on("cameraOpen", (playCallBackInfo: any) => {
        this.overlayManager.updateOverlay(playCallBackInfo.width, playCallBackInfo.height);
      });
      this.enhancer.on("cameraClose", (playCallBackInfo: any) => {
        console.log(playCallBackInfo.deviceId);
      });


      this.isLoaded = true;
      await this.normalizer.setRuntimeSettings(Template.color);
    })();
  }
}

After instantiating the Dynamsoft Camera Enhancer, we need to bind it to a div element for displaying the camera stream:

let uiElement = document.getElementById('videoContainer');
if (uiElement) {
  await this.enhancer.setUIElement(uiElement);
}

If you have multiple cameras, you can call the getAllCameras method to get the camera list:

let cameras = await this.enhancer.getAllCameras();
this.listCameras(cameras);

Then select a camera and open it:

async openCamera(): Promise<void> {
  if (this.videoSelect) {
    let deviceId = this.videoSelect.value;
    if (this.enhancer) {
      await this.enhancer.selectCamera(this.cameraInfo[deviceId]);
      await this.enhancer.open()
    }
  }

}

The detectQuad method supports different input types. In the above section, we used the detectQuad method to detect the document from an image file. Now we use it to detect the document from frames of the camera stream.

detect(): void {
  if (this.normalizer && this.enhancer && this.isDetecting) {

    let data = this.enhancer.getFrame().toCanvas();
    this.normalizer.detectQuad(data).then((results: any) => {
      this.overlayManager.clearOverlay();
      if (!this.isDetecting) return;
      try {
        if (results.length > 0) {
          if (this.captured.length > 0) {
            this.captured.pop();
          }

          let result = results[0];
          let points = result['location']['points'];
          this.captured.push({ 'image': data, 'points': points });
          this.overlayManager.drawOverlay(
            points,
          );
        }
      } catch (e) {
        alert(e);
      }
      this.detect();
    });
  }
}

The document detection job runs in a web worker. Thus, don't call it continuously. You must wait for the previous detection job to finish before calling it again.

Run ng serve to have fun with the web document detection and recification app.

Source Code

https://github.com/yushulx/angular-barcode-mrz-document-scanner

Blog