How to Implement Machine Readable Zone (MRZ) Recognition in Python

Machine Readable Zone (MRZ) is a crucial feature adopted in modern passports, visas, and ID cards. It contains essential information about the document holder, such as their name, gender, country code, and document number. MRZ recognition plays a key role in border control, airport security, and hotel check-in processes. In this tutorial, we will demonstrate how to leverage the Dynamsoft Capture Vision SDK to implement MRZ recognition across Windows, Linux, and macOS platforms. This guide will provide a step-by-step approach to harness the SDK’s powerful features, making cross-platform MRZ detection seamless and efficient.

Python MRZ Recognition Demo on macOS

Prerequisites

Dynamsoft Capture Vision Trial License: Obtain a 30-Day trial license key for the Dynamsoft Capture Vision SDK.
Python Packages: Install the required Python packages using the following commands:
```
pip install dynamsoft-capture-vision-bundle opencv-python
```
What are these packages for?
- dynamsoft-capture-vision-bundle is the Dynamsoft Capture Vision SDK for Python.
- opencv-python captures camera frames and displays processed image results.

Getting Started with the Dynamsoft Python Capture Vision Example

The official MRZ scanner example demonstrates how to create a simple Python-based MRZ reader using the Dynamsoft Capture Vision SDK in a short time.

Let's take a look at the source code and analyze its functionality:


import sys
from dynamsoft_capture_vision_bundle import *
import os

class MRZResult:
    def __init__(self, item: ParsedResultItem):
        self.doc_type = item.get_code_type()
        self.raw_text=[]
        self.doc_id = None
        self.surname = None
        self.given_name = None
        self.nationality = None
        self.issuer = None
        self.gender = None
        self.date_of_birth = None
        self.date_of_expiry = None
        if self.doc_type == "MRTD_TD3_PASSPORT":
            if item.get_field_value("passportNumber") != None and item.get_field_validation_status("passportNumber") != EnumValidationStatus.VS_FAILED:
                self.doc_id = item.get_field_value("passportNumber")
            elif item.get_field_value("documentNumber") != None and item.get_field_validation_status("documentNumber") != EnumValidationStatus.VS_FAILED:
                self.doc_id = item.get_field_value("documentNumber")

        line = item.get_field_value("line1")
        if line is not None:
            if item.get_field_validation_status("line1") == EnumValidationStatus.VS_FAILED:
                line += ", Validation Failed"
            self.raw_text.append(line)
        line = item.get_field_value("line2")
        if line is not None:
            if item.get_field_validation_status("line2") == EnumValidationStatus.VS_FAILED:
                line += ", Validation Failed"
            self.raw_text.append(line)
        line = item.get_field_value("line3")
        if line is not None:
            if item.get_field_validation_status("line3") == EnumValidationStatus.VS_FAILED:
                line += ", Validation Failed"
            self.raw_text.append(line)

        if item.get_field_value("nationality") != None and item.get_field_validation_status("nationality") != EnumValidationStatus.VS_FAILED:
            self.nationality = item.get_field_value("nationality")
        if item.get_field_value("issuingState") != None and item.get_field_validation_status("issuingState") != EnumValidationStatus.VS_FAILED:
            self.issuer = item.get_field_value("issuingState")
        if item.get_field_value("dateOfBirth") != None and item.get_field_validation_status("dateOfBirth") != EnumValidationStatus.VS_FAILED:
            self.date_of_birth = item.get_field_value("dateOfBirth")
        if item.get_field_value("dateOfExpiry") != None and item.get_field_validation_status("dateOfExpiry") != EnumValidationStatus.VS_FAILED:
            self.date_of_expiry = item.get_field_value("dateOfExpiry")
        if item.get_field_value("sex") != None and item.get_field_validation_status("sex") != EnumValidationStatus.VS_FAILED:
            self.gender = item.get_field_value("sex")
        if item.get_field_value("primaryIdentifier") != None and item.get_field_validation_status("primaryIdentifier") != EnumValidationStatus.VS_FAILED:
            self.surname = item.get_field_value("primaryIdentifier")
        if item.get_field_value("secondaryIdentifier") != None and item.get_field_validation_status("secondaryIdentifier") != EnumValidationStatus.VS_FAILED:
            self.given_name = item.get_field_value("secondaryIdentifier")
    def to_string(self):
        msg = (f"Raw Text:\n")
        for index, line in enumerate(self.raw_text):
            msg += (f"\tLine {index + 1}: {line}\n")
        msg+=(f"Parsed Information:\n"
            f"\tDocumentType: {self.doc_type or ''}\n"
            f"\tDocumentID: {self.doc_id or ''}\n"
            f"\tSurname: {self.surname or ''}\n"
            f"\tGivenName: {self.given_name or ''}\n"
            f"\tNationality: {self.nationality or ''}\n"
            f"\tIssuingCountryorOrganization: {self.issuer or ''}\n"
            f"\tGender: {self.gender or ''}\n"
            f"\tDateofBirth(YYMMDD): {self.date_of_birth or ''}\n"
            f"\tExpirationDate(YYMMDD): {self.date_of_expiry or ''}\n")
        return msg
def print_results(result: ParsedResult) -> None:
    tag = result.get_original_image_tag()
    if isinstance(tag, FileImageTag):
        print("File:", tag.get_file_path())
    if result.get_error_code() != EnumErrorCode.EC_OK:
        print("Error:", result.get_error_string())        
    else:
        items = result.get_items()
        print("Parsed", len(items), "MRZ Zones.")
        for item in items:
            mrz_result = MRZResult(item)
            print(mrz_result.to_string())

if __name__ == '__main__':

    print("**********************************************************")
    print("Welcome to Dynamsoft Capture Vision - MRZ Sample")
    print("**********************************************************")

    error_code, error_message = LicenseManager.init_license("LICENSE-KEY")
    if error_code != EnumErrorCode.EC_OK and error_code != EnumErrorCode.EC_LICENSE_CACHE_USED:
        print("License initialization failed: ErrorCode:", error_code, ", ErrorString:", error_message)
    else:
        cvr_instance = CaptureVisionRouter()
        while (True):
            image_path = input(
                ">> Input your image full path:\n"
                ">> 'Enter' for sample image or 'Q'/'q' to quit\n"
            ).strip('\'"')

            if image_path.lower() == "q":
                sys.exit(0)

            if image_path == "":
                image_path = "../Images/passport-sample.jpg"

            if not os.path.exists(image_path):
                print("The image path does not exist.")
                continue
            result = cvr_instance.capture(image_path, "ReadPassportAndId")
            if result.get_error_code() != EnumErrorCode.EC_OK:
                print("Error:", result.get_error_code(), result.get_error_string())
            else:
                parsed_result = result.get_parsed_result()
                if parsed_result is None or len(parsed_result.get_items()) == 0:
                    print("No parsed results.")
                else:
                    print_results(parsed_result)
    input("Press Enter to quit...")

Explanation

The LicenseManager.init_license method initializes the Dynamsoft Capture Vision SDK with a valid license key.
The CaptureVisionRouter class manages image processing tasks and coordinates various image processing modules. Its capture method processes the input image and returns the result.
The ReadPassportAndId is a built-in template specifying the processing mode. The SDK supports various processing modes, such as MRZ recognition, document edge detection, and barcode detection.
The get_parsed_result method retrieves the MRZ recognition result as a dictionary. The MRZResult class extracts and wraps the relevant MRZ information. Since this class can be reused across different applications, it is recommended to move it to a utils.py file.

In the next section, we will use OpenCV to visualize the MRZ recognition results and display the detected MRZ zones on the passport image.

Visualizing Machine Readable Zone Location in a Passport Image

In the code above, result is an instance of the CapturedResult class. Calling its get_recognized_text_lines_result() method retrieves a list of TextLineResultItem objects. Each TextLineResultItem object contains the coordinates of the detected text line. Use the following code snippet to extract the coordinates and draw contours on the passport image:

cv_image = cv2.imread(image_path)
line_result = result.get_recognized_text_lines_result()

items = line_result.get_items()
for item in items:
    location = item.get_location()
    x1 = location.points[0].x
    y1 = location.points[0].y
    x2 = location.points[1].x
    y2 = location.points[1].y
    x3 = location.points[2].x
    y3 = location.points[2].y
    x4 = location.points[3].x
    y4 = location.points[3].y
    del location

    cv2.drawContours(
        cv_image, [np.intp([(x1, y1), (x2, y2), (x3, y3), (x4, y4)])], 0, (0, 255, 0), 2)

cv2.imshow(
    "Original Image with Detected MRZ Zone", cv_image)
cv2.waitKey(0)
cv2.destroyAllWindows()

Scanning and Recognizing MRZ in Real-time via Webcam

Scanning and recognizing MRZ in real-time via webcam requires capturing a continuous image stream. We can use the OpenCV library to capture frames from the webcam and process them with the Dynamsoft Capture Vision SDK. The following code snippet demonstrates how to implement real-time MRZ recognition using a webcam:

from dynamsoft_capture_vision_bundle import *
import cv2
import numpy as np
import queue
from utils import *


class FrameFetcher(ImageSourceAdapter):
    def has_next_image_to_fetch(self) -> bool:
        return True

    def add_frame(self, imageData):
        self.add_image_to_buffer(imageData)


class MyCapturedResultReceiver(CapturedResultReceiver):
    def __init__(self, result_queue):
        super().__init__()
        self.result_queue = result_queue

    def on_captured_result_received(self, captured_result):
        self.result_queue.put(captured_result)


if __name__ == '__main__':
    errorCode, errorMsg = LicenseManager.init_license(
        "LICENSE-KEY")
    if errorCode != EnumErrorCode.EC_OK and errorCode != EnumErrorCode.EC_LICENSE_CACHE_USED:
        print("License initialization failed: ErrorCode:",
              errorCode, ", ErrorString:", errorMsg)
    else:
        vc = cv2.VideoCapture(0)
        if not vc.isOpened():
            print("Error: Camera is not opened!")
            exit(1)

        cvr = CaptureVisionRouter()
        fetcher = FrameFetcher()
        cvr.set_input(fetcher)

        # Create a thread-safe queue to store captured items
        result_queue = queue.Queue()

        receiver = MyCapturedResultReceiver(result_queue)
        cvr.add_result_receiver(receiver)

        errorCode, errorMsg = cvr.start_capturing("ReadPassportAndId")

        if errorCode != EnumErrorCode.EC_OK:
            print("error:", errorMsg)

        while True:
            ret, frame = vc.read()
            if not ret:
                print("Error: Cannot read frame!")
                break

            fetcher.add_frame(convertMat2ImageData(frame))

            if not result_queue.empty():
                captured_result = result_queue.get_nowait()

                items = captured_result.get_items()
                for item in items:

                    if item.get_type() == EnumCapturedResultItemType.CRIT_TEXT_LINE:
                        text = item.get_text()
                        line_results = text.split('\n')
                        location = item.get_location()
                        x1 = location.points[0].x
                        y1 = location.points[0].y
                        x2 = location.points[1].x
                        y2 = location.points[1].y
                        x3 = location.points[2].x
                        y3 = location.points[2].y
                        x4 = location.points[3].x
                        y4 = location.points[3].y
                        cv2.drawContours(
                            frame, [np.intp([(x1, y1), (x2, y2), (x3, y3), (x4, y4)])], 0, (0, 255, 0), 2)

                        delta = y3 - y1
                        for line_result in line_results:
                            cv2.putText(
                                frame, line_result, (x1, y1), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 1, cv2.LINE_AA)
                            y1 += delta

                        del location

                    elif item.get_type() == EnumCapturedResultItemType.CRIT_PARSED_RESULT:
                        mrz_result = MRZResult(item)
                        print(mrz_result.to_string())

            if cv2.waitKey(1) & 0xFF == ord('q'):
                break

            cv2.imshow('frame', frame)

        cvr.stop_capturing()
        vc.release()
        cv2.destroyAllWindows()

Explanation

The FrameFetcher class implements the ImageSourceAdapter interface to feed frame data into the built-in buffer.
The MyCapturedResultReceiver class implements the CapturedResultReceiver interface. The on_captured_result_received method runs on a native C++ worker thread, sending CapturedResult objects to the main thread where they are stored in a thread-safe queue for further use.
A CapturedResult contains several CapturedResultItem objects. The CRIT_TEXT_LINE type represents recognized text lines, while the CRIT_PARSED_RESULT type represents parsed MRZ data.

Running the Real-time MRZ Recognition Demo on Windows

Source Code

https://github.com/yushulx/python-mrz-scanner-sdk/tree/main/examples/official

Blog