cross-post from medium.com

A low-code approach to incorporating machine learning into the Internet of Things

Gartner has predicted the number of connected devices will rise to over 25 billion by 2021. And given the variety of devices out there, getting started with IoT can appear daunting. Setting up communication with or between these devices is often non-trivial. Further challenges arise if you want to integrate machine learning! Solutions need to pull together different device APIs, services, and sometimes protocols.

Node-RED with TensorFlow.js brings machine learning to IoT in an easy, low-code way. It opens new, creative approaches to enable machine learning for the Internet of Things. Image recognition, audio classification, etc. all possible on device with minimal code.

Enter Node-RED

Node-RED is a flow-based visual programming tool. With its browser-based editor you can simply wire together hardware devices, APIs and online services to create your application. You develop powerful applications by connecting nodes instead of writing code. And you can deploy them with a single click.

Node-RED runs on local workstations, the cloud, and edge devices. It has become an ideal tool for the Raspberry Pi and other low cost hardware.

The Node-RED runtime is lightweight and built on top of Node.js. It takes full advantage of Node.js’ event-driven, non-blocking I/O model. There is also the added benefit of tapping into the most used programming language — JavaScript!

With all the Node-RED community resources and vast NPM ecosystem, you can create IoT flows in a user-friendly manner that are imaginative and help save lives.

Hello TensorFlow.js

TensorFlow.js is an open source JavaScript library. It permits you to build, train, and run machine learning models in the browser and Node.js.

Often, enabling AI capabilities involves sending the data from a device to a server. The calculations happen on the server and the results returned to the device to take action. This is not ideal when data security or network reliability is a problem.

But, with TensorFlow.js there is an increase in privacy and data security. The data does not leave the device! Training and predictions can happen directly on the device collecting the data. This also makes it possible to run offline and in remote locations with no connectivity.

Node-RED meet TensorFlow.js

The combination of Node-RED and TensorFlow.js means you can build IoT apps that use machine learning simply by dragging and dropping. Drag and drop a machine learning node, wire it up, and deploy to your device.

But to get to that point you need to have the TensorFlow.js nodes available. TensorFlow.js nodes are starting to show up in the Node-RED library and across GitHub repos and more are being released regularly. These nodes provide various machine learning functionality to add to your flow. But what if there isn’t a TensorFlow.js node for your machine learning task? You can create it!

The extensibility of Node-RED allows you to create custom nodes for your needs. Packing Node-RED nodes is similar to packaging Node.js modules, but with some extra information.

A Node-RED node consists of three main files:

• package.json: standard file used by Node.js modules, but with an added node-red section
• JavaScript file that defines the node’s behavior
• HTML file that defines the node’s properties, edit dialog and help text

The JavaScript file is where you would wrap your TensorFlow.js code. It would load the TensorFlow.js model and run the prediction.

Once bundled the custom node is available to wire into a flow and deploy.

There may be challenges

As straightforward as it may appear, there can still be challenges and concerns to keep in mind.

Since, you are dealing with edge devices performance is an utmost priority. Models may be too big to load onto an edge device. Or may require specific optimizing to perform well in JavaScript.

Also, when in the node’s life-cycle should you load the model? Should you have a single node to process input/output and run prediction? Or split the work across two or three nodes?

The TensorFlow.js model you are using and the specific use case it addresses often dictate the approach and answers to a lot of these concerns.

Getting started

Combining TensorFlow.js with Node-RED lowers the barrier to entry into machine learning. From the drag-and-drop interface to the one click deploy, IoT enthusiasts and developers can incorporate machine learning in an accessible and rapid manner.

Visit the Node-RED and TensorFlow.js code pattern to check out a sample solution. Learn more by viewing the full code and deploying the tutorial.

Check out the Node-RED library for more TensorFlow.js nodes, flows, and collections as they become available. Anyone is welcome to contribute there so that others can learn from your work.