How to Build Mobile Check Capture App with Xamarin.Forms and Dynamsoft Document SDK

A Mobile Check Capture app, also known as a Check Deposit app or Mobile Check Deposit app, is a mobile application that allows users to deposit checks using their smartphones or tablets. It provides a convenient way for individuals and businesses to deposit checks without visiting a bank branch or ATM. Typically, a Mobile Check Capture app utilizes the camera on a mobile device to capture images of the front and back of a check. This article will show how to build a mobile check capture app for both Andorid and iOS platforms using Xamarin.Forms and Dynamsoft Document Normalizer.

Video Demo

NuGet Packages

Dynamsoft.DocumentNormalizer.Xamarin.Forms

Steps to Build a Mobile Check Capture App

In the following sections, we will guide you through the steps to build a mobile check capture app. The functionalities of the app include:

Capture images of the front and back of a check from camera stream.
Detect the check edges.
Edit the quadrilateral area of the check.

Step 1: Create a Xamarin.Forms Project

In Visual Studio 2022, create a new project using the Mobile App (Xamarin.Forms) template.
Open NuGet Package Manager to install Dynamsoft.DocumentNormalizer.Xamarin.Forms:

Configure Info.plist for iOS and AndroidManifest.xml for Android.

iOS Info.plist

<key>NSCameraUsageDescription</key>
<string>This app is using the camera</string>

Android AndroidManifest.xml

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android" android:versionCode="1" android:versionName="1.0" package="com.companyname.documentscanner">
    <uses-sdk android:minSdkVersion="21" android:targetSdkVersion="31" />
    <application android:label="DocumentScanner.Android" android:theme="@style/MainTheme"></application>
    <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
</manifest>

Step 2: Initialize Dynamsoft Document Normalizer

Apply for a trial license of Dynamsoft Document Normalizer.

Write platform-specific code to initialize Dynamsoft Document Normalizer.

iOS AppDelegate.cs

public override bool FinishedLaunching(UIApplication app, NSDictionary options)
{
    global::Xamarin.Forms.Forms.Init();

    App.ScreenWidth = UIScreen.MainScreen.Bounds.Width;
    App.ScreenHeight = UIScreen.MainScreen.Bounds.Height;

    LoadApplication(new App(new DCVCameraEnhancer(), new DCVDocumentNormalizer(), new DCVLicenseManager()));

    return base.FinishedLaunching(app, options);
}

Android MainActivity.cs

protected override void OnCreate(Bundle savedInstanceState)
{
    base.OnCreate(savedInstanceState);

    Xamarin.Essentials.Platform.Init(this, savedInstanceState);
    global::Xamarin.Forms.Forms.Init(this, savedInstanceState);

    var width = Resources.DisplayMetrics.WidthPixels;
    var height = Resources.DisplayMetrics.HeightPixels;
    var density = Resources.DisplayMetrics.Density;

    App.ScreenWidth = (width - 0.5f) / density;
    App.ScreenHeight = (height - 0.5f) / density;

    LoadApplication(new App(new DCVCameraEnhancer(this), new DCVDocumentNormalizer(), new DCVLicenseManager(this)));
}

Pass the instancess of ICameraEnhancer, IDocumentNormalizer and ILicenseManager to the constructor of App in App.xaml.cs.

public static ICameraEnhancer dce;
public static IDocumentNormalizer ddn;

public static double ScreenWidth;
public static double ScreenHeight;

public App(ICameraEnhancer enhancer, IDocumentNormalizer normalizer, ILicenseManager manager)
{
    InitializeComponent();

    dce = enhancer;
    ddn = normalizer;

    MainPage = new NavigationPage(new MainPage(manager));
}

Set and verify the license in MainPage.xaml.cs.

public partial class MainPage : ContentPage, ILicenseVerificationListener
{
    private ILicenseManager licenseManager;
    private bool isLicenseValid = true;

    public MainPage(ILicenseManager licenseManager)
    {
        InitializeComponent();
        this.licenseManager = licenseManager;
        licenseManager.InitLicense("DLS2eyJoYW5kc2hha2VDb2RlIjoiMjAwMDAxLTE2NDk4Mjk3OTI2MzUiLCJvcmdhbml6YXRpb25JRCI6IjIwMDAwMSIsInNlc3Npb25QYXNzd29yZCI6IndTcGR6Vm05WDJrcEQ5YUoifQ==", this);
    }

    public void LicenseVerificationCallback(bool isSuccess, string msg)
    {
        if (!isSuccess)
        {
            Device.BeginInvokeOnMainThread(async () => {
                isLicenseValid = false;
                await DisplayAlert("Error", msg, "OK");
            });
        }
    }
}

The LicenseVerificationCallback() is invoked in a background thread. To update the UI, we need to use Device.BeginInvokeOnMainThread().

Step 3: Design the UI of the Main Page

There are three Xamarin.Forms widgets used on the main page:

Image: Display a static PNG image built as embedded resource.
Label: Display some description text.
Button: Navigate to the check information page.

How to load an embedded resource image in Xamarin.Forms?

Add an image file to the project and set its Build Action to Embedded resource.

Create an ImageResourceExtension class to load embedded resource images.

// You exclude the 'Extension' suffix when using in Xaml markup
[Preserve(AllMembers = true)]
[ContentProperty(nameof(Source))]
public class ImageResourceExtension : IMarkupExtension
{
    public string Source { get; set; }

    public object ProvideValue(IServiceProvider serviceProvider)
    {
        if (Source == null)
            return null;

        var imageSource = ImageSource.FromResource(Source, typeof(ImageResourceExtension).GetTypeInfo().Assembly);

        return imageSource;
    }
}

In a XAML file, import the namespace of local and load a PNG file as follows:

<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
            xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
            xmlns:local="clr-namespace:DocumentScanner;assembly=DocumentScanner" 
            x:Class="DocumentScanner.MainPage"
            BackgroundColor="Transparent"
            Title="CheckCapture">

    <StackLayout Margin="20,20,20,20" >
        <Image Source="{local:ImageResource DocumentScanner.icon-cover.png}"/>
    </StackLayout>

</ContentPage>

The local:ImageResource can be used to set an image source for a button as well:

<Button x:Name="customRenderer" Text="Deposit Check" TextColor="White" HorizontalOptions="Center"  Clicked="OnCustomRendererButtonClicked" BackgroundColor="Orange" ImageSource="{local:ImageResource DocumentScanner.icon-capture.png}" HeightRequest="50" WidthRequest="200"/>

Step 4: Design the UI of the Check Information Page

The check information page contains the following widgets:

TitleView: A custom title view that contains a title label and an upload button.

<NavigationPage.TitleView>
    <StackLayout Orientation="Horizontal" Margin="0,0,10,0">
        <Label Text="Check" TextColor="white" FontAttributes="Bold" VerticalOptions="Center" HorizontalOptions="StartAndExpand"/>
        <Button
            x:Name="upload"
            Clicked="OnUploadClicked"
            BackgroundColor="transparent"  HeightRequest="50" WidthRequest="50" ImageSource="{local:ImageResource DocumentScanner.icon-upload.png}" VerticalOptions="Center">
        </Button>
    </StackLayout>
</NavigationPage.TitleView>

Editor: Input the check amount.

<StackLayout Orientation="Horizontal" Margin="20,0,20,20">
    <Label Text="Amount: $" FontAttributes="Bold"
VerticalOptions="Center" />

    <Editor x:Name="amount" HorizontalOptions="FillAndExpand" Placeholder="" VerticalOptions="Center"/>
</StackLayout>

Frame: Create a layout with rounded corners.

<Frame CornerRadius="10" Margin="20,20,20,20">
    <StackLayout HeightRequest="250" WidthRequest="400" BackgroundColor="White" >
    </StackLayout>
</Frame>

Image: Display the captured check image. The image is clickable for launching the editor page.

<Image x:Name="front_image" Aspect="AspectFit">
    <Image.GestureRecognizers>
        <TapGestureRecognizer Tapped="FrontImageTapped" />
    </Image.GestureRecognizers>
</Image>

Label: Display some description text.
Button: Launch the check capture page.

To update the image source after capturing or editing image quadrilaterals from another content page, you can use MessagingCenter to handle the communication between the pages.

In the check info page, subscribe to the ImageData message. The CustomRendererPage is the check capture page.

MessagingCenter.Subscribe<CustomRendererPage, InfoData>(this, "ImageData", (sender, arg) =>
{
    App.ddn.InitRuntimeSettings(Templates.color);

    NormalizedImageResult normalizedImage = App.ddn.Normalize(arg.imageData, arg.quad);

    if (isFront)
    {
        _frontData = arg;
        front_image.Source = normalizedImage.image.ToImageSource();

        if (Device.RuntimePlatform == Device.iOS)
        {
            front_image.RotateTo(normalizedImage.image.orientation + 180);
        }
        else
        {
            front_image.RotateTo(270);
        }
    }
    else
    {
        _backData = arg;
        back_image.Source = normalizedImage.image.ToImageSource();

        if (Device.RuntimePlatform == Device.iOS)
        {
            back_image.RotateTo(normalizedImage.image.orientation + 180);
        }
        else
        {
            back_image.RotateTo(270);
        }
    }
});

In the CustomRendererPage, send the image data to the check info page:

public void DetectResultCallback(int id, ImageData imageData, DetectedQuadResult[] quadResults)
{
    if (imageData != null && quadResults != null)
    {
        Device.BeginInvokeOnMainThread(async () => {
            ImageData data = new ImageData();
            data.imageSource = imageData.imageSource;
            data.bytes = new List<byte>(imageData.bytes);
            data.width = imageData.width;
            data.height = imageData.height;
            data.stride = imageData.stride;
            data.format = imageData.format;
            data.orientation = imageData.orientation;

            InfoData info = new InfoData();
            info.imageData = data;
            info.quad = quadResults[0].Location;

            MessagingCenter.Send(this, "ImageData", info);

            await Navigation.PopAsync();

        });
    }
}

Step 5: Capture Images of the Front and Back of a Check

The CustomRendererPage is used to recognize document edges from camera stream in real-time. To display the camera stream, add the DCVCameraView widget. Users should hold the screen in landscape mode to get better capture experience. Therefore, label and button are both rotated 90 degree.

<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             xmlns:dynamsoft = "clr-namespace:DDNXamarin;assembly=DDN-Xamarin"
             xmlns:local="clr-namespace:DocumentScanner;assembly=DocumentScanner" 
             x:Class="DocumentScanner.CustomRendererPage"
             Title="Scan Check">
    <ContentPage.Content>
        <AbsoluteLayout>
            <dynamsoft:DCVCameraView AbsoluteLayout.LayoutBounds="0,0,1,1" AbsoluteLayout.LayoutFlags="All" x:Name="preview">
            </dynamsoft:DCVCameraView>
            <Label Text="Please use landscape mode" Rotation="90" TextColor="White" AbsoluteLayout.LayoutBounds="0.7,0.5,300,300" AbsoluteLayout.LayoutFlags="PositionProportional"/>

            <Button x:Name="capture" 
                    AbsoluteLayout.LayoutBounds="0.5,0.95,80,80" AbsoluteLayout.LayoutFlags="PositionProportional"
                    Clicked="OnButtonClicked" ImageSource="{local:ImageResource DocumentScanner.icon-capture.png}" Rotation="90" BackgroundColor="Orange" BorderRadius="40" 
        >
            </Button>
        </AbsoluteLayout>
    </ContentPage.Content>
</ContentPage>

The button triggers DetectResultCallback when the document edges are found.

void OnButtonClicked(object sender, EventArgs e)
{
    App.ddn.EnableReturnImageOnNextCallback();
}

Step 6: Edit the Quadrilateral Area of the Check

The editor page contains a DCVImageEditorView, which supports changing and saving the four corners of the document.

<?xml version="1.0" encoding="utf-8" ?>
<ContentPage xmlns="http://xamarin.com/schemas/2014/forms"
             xmlns:x="http://schemas.microsoft.com/winfx/2009/xaml"
             xmlns:dynamsoft = "clr-namespace:DDNXamarin;assembly=DDN-Xamarin"
              xmlns:local="clr-namespace:DocumentScanner;assembly=DocumentScanner" 
             x:Class="DocumentScanner.QuadEditorPage">
    <NavigationPage.TitleView>
        <StackLayout Orientation="Horizontal" Margin="0,0,10,0">
            <Label Text="Edit and Save Quad" TextColor="white" FontAttributes="Bold" VerticalOptions="Center" HorizontalOptions="StartAndExpand"/>
            <Button
                x:Name="normalize"
                Clicked="OnNormalizeClicked"
                BackgroundColor="transparent"  HeightRequest="50" WidthRequest="50" ImageSource="{local:ImageResource DocumentScanner.icon-save.png}" VerticalOptions="Center">
            </Button>
        </StackLayout>
    </NavigationPage.TitleView>

    <ContentPage.Content>
        <AbsoluteLayout>
            <dynamsoft:DCVImageEditorView AbsoluteLayout.LayoutBounds="0,0,1,1" AbsoluteLayout.LayoutFlags="All"
                x:Name="imageEditor">
            </dynamsoft:DCVImageEditorView>

        </AbsoluteLayout>

    </ContentPage.Content>
</ContentPage>

In the check info page, send the image and the detected quadrilateral to the editor page:

private void BackImageTapped(object sender, EventArgs e)
{

    if (_backData != null) {
        DetectedQuadResult result = new DetectedQuadResult();
        result.Location = _backData.quad;
        Navigation.PushAsync(new QuadEditorPage(_backData.imageData,  new DetectedQuadResult[] {result}));
    }

}

In the editor page, get the updated quadrilateral and pass it back to the info page via MessagingCenter:

async void OnNormalizeClicked(object sender, EventArgs e)
{
    try
    {
        var quad = imageEditor.getSelectedQuadResult();
        if (quad != null)
        {
            InfoData data = new InfoData();
            data.imageData = this.data;
            data.quad = quad;
            MessagingCenter.Send(this, "ImageData", data);
            await Navigation.PopAsync();
        }
    }
    catch (Exception exception)
    {
        Device.BeginInvokeOnMainThread(async () => {
            await DisplayAlert("Error", exception.ToString(), "OK");
        });
    }
}

Source Code

https://github.com/yushulx/Xamarin-forms-document-scanner/tree/checkcapture

Blog