In 2020, facial recognition became one of the most talked about technologies due to its wide adoption and highly controversial use by the Chinese government. However, the use of facial recognition technologies is not limited to government surveillance. Proprietary implementations of facial recognition are built into many modern smartphones and Windows laptops to facilitate seamless biometric identification and authentication of the user. Facial recognition is commonly used in the entertainment and fashion industries, security and surveillance applications.

When it comes to facial recognition libraries, the number of available options wide. You have the choice of using one of the open-source libraries of taking the proprietary route and licensing a face recognition SDK from a specialized company. In the end, you may be able to build an app using either platform, yet the path and the end result may differ significantly. So let’s discuss the available options and see their pros and contras.

The open source libraries

There are several open source facial recognition project around, such as OpenBR, Flandmark, OpenFaceTracker, Face-api.js. However, the most popular projects are OpenCV and dlib.

These two projects solve similar yet different tasks. While OpenCV mostly offers image processing with some machine learning available if you build from the source, Dlib is a designated AI-based machine learning library. Since these are different libraries solving different tasks, your project may need either one or both of them.

Each of these projects has a community backing the development. When it comes purely to facial recognition, both libraries do their job, yet neither is perfect. You can read a comprehensive comparison of the various facial recognition methods used in OpenCV and Dlip in Face Detection - OpenCV, Dlib and Deep Learning | Learn OpenCV. Without much ado, here’s the brief summary.

Pros:

• Lightweight
• Ready to go
• Free and open source
• Extensive communities and forum support

Cons:

• Accuracy suffers in all modes
• Difficult to strike the right balance between acceptable false negative and false positive rates
• Backend architecture design and development required
• Some features only work on frontal images
• Ongoing issues recognizing images of visible minorities

Facial recognition SDK

If you decide to follow the proprietary route, you’ll have several options from the two major categories: offline facial recognition SDKs and online APIs. Let’s start with the former.

Commercial facial recognition SDKs are distributed as sets of libraries for various platforms and in various programming languages. The most common SDK’s handle the entire recognition process locally. Nothing is being transmitted over the network, and the recognition makes use of the local computer’s resources for doing its work. While this is can be a significant benefit in some applications, you have to consider the costs of designing, implementing and securing the backend infrastructure, which includes a database. An example of facial recognition SDK is Luxand FaceSDK.

Pros:

• Works almost real-time on a local CPU
• Easier to implement compared to open source libraries
• Highly accurate recognition
• Unlike online API’s, can be used inside DMZ
• Good choice for high-power computers

Cons:

• High upfront costs (must purchase a license before starting development)
• Licensing costs (sometimes recurrent)
• More difficult to implement compared to online API’s
• Infrastructure design and implementation required (including database architecture)
• Depending on the SDK, limited support for different programming languages

Facial recognition API

Some SDKs expose high-level APIs implementing common tasks associated with face recognition on a remote server.

The main advantage of facial recognition APIs is the ease of implementing them in your product. The backend infrastructure has been already taken care for by the API provider, so your app or Web service only needs to make a few calls to get things done. Since the APIs are making calls to a remote server instead of including complex biometric models and machine learning stuff, the implementation is extremely lightweight. Authenticating, registering a face and submitting a face for recognition/identification are implemented with several high-level calls.

Facial recognition APIs have their share of drawbacks, the most important is the fact that you need an Internet connection for it to work; this, however, has very little importance for Web apps and apps having access to the Internet.

Pros:

• Zero upfront costs
• Extremely lightweight
• The easiest way to start with
• Complex operation implemented via single high-level API call
• Everything is ready
• Infrastructure development not required (the database is already in the cloud)
• All programming languages supported
• Good choice for web applications and thin clients
• Good choice for mobile apps

Cons:

• Recurring costs or licensed per usage
• Online only, Internet connection always required
• Higher latency due to online overhead

Conclusion

The choice of a free open-source SDK, a commercial version of the same, and an online facial recognition API depends on the intended application, available time and resources.

Open Source libraries are free of charge, yet require the most effort to implement. Commercial SDKs can be easier to implement and deliver more convincing results in the end, yet you still have to design and implement the backend on your own.

The online APIs by and large are the easiest to use, requiring no upfront fees and only charging for services that you actually use. In my view, online APIs are great for connected mobile apps and Web apps that aren’t time critical, with expected small to medium usage. For my apps, I prefer Luxand.cloud API for its fast implementation, precisely tuned recognition and ease of use.