Machine learning implemented in JavaScript with ml5.js

Introduction

I'm glad there are machine learning implementations in JavaScript. Abstracts a lot of the complexities associated with training & using machine learning models. I just hope ml5.js doesn't become obsolete tomorrow or the next month! Python has been the modern de-facto language for #machinelearning. And there are tons of python libraries to implement machine learning tasks. However, JavaScript implementations of machine learning are fairly recent.

I chose this ml5.js for my "SailorMoon Pose Sign/Login"project because you can start just by making a reference to the library via a CDN and use it's inbuilt methods. Similar to making a reference to other libraries such as JQuery. There is no installation required.

Include ml5.js in project

<head>
<script src="https://unpkg.com/ml5@0.1.1/dist/ml5.min.js">
</script>
</head>

ml5.js

ml5.js is a browser based wrapper around tensorflow.js. It describes itself as "friendly machine learning for the web". ml5.js provides pre-trained models in the browser. If you were attempting machine learning from scratch you would probably train a model on your own. So for example feed lots of input(for example human poses) to a neural network. And, over time the neural network due to receiving simulation/lots of input becomes sophisticated enough to give a predictable intelligible output due to recognition of patterns. This is akin to synapses in the brain that get strengthened over time with repeated stimuli. This article elaborates a bit further on the technical side of things.

Pre-trained models such as MobileNet which is used to classify images are used with ml5.js by passing it as an argument to the ml5.ImageClassifier() method of the ml5object, along with a callback function:

ml5.imageClassifier(MobileNet,callbackFunction);

The MobileNet model has been trained with 15 million images so you can image how useful it is to use a pre-trained model rather than create one of your own. Ml5.js references this library in the cloud, and therefore you can use it in your own projects.

Some other pre-trained models available for you to use with ml5.js are:

PoseNet (to detect poses)
BodyPix
FaceAPI
YOLO
And others

While using ml5.js it's important to note that: "ml5.js supports both error-first callbacks and Promises in all methods.". Let's deconstruct:

Error-first callback
Promise

Error first callback

A callback function is a function that is executed aka called-back after another function has finished executing. In JavaScript a function is an object. Which means you can pass a function as an argument to another function. This function that receives the callback function as an argument is called a higher order function.

By using callback functions you ensure that a block of code contained within the callback function doesn't execute unless it's parent function runs. This idea helps us create asynchronous code/applications.

Read this worksheet if you'd like to practice/learn more about callbacks:

common syntax/example is:

function main(argument1, callbackFxn(){
}
}//end of main function

Or you can reference a named callback function:

function main(argument1, hello());

ml5.js which uses an error-first callback pattern to handle asynchronicity. The callback function will be executed once the async operation ends with the 1st argument being an error object. It looks like so:

ml5.methodName(ModelName, function(errorObj, someData)
{if(errorObj){
//Error executes first
} 
//some Data is returned from the pre-trained model
});

If no error occurs, then errorObj object is set to null and some data is returned.

Promises

Promises in JavaScript are used to handle asynchronous operations. They are objects that reference a value to be received in the future. Promises were introduced as part of the ES6 specification. Async operations as you read above can be accomplished by callback functions. However,nesting of multiple callback functions is messy and can cause what is known as callback hell. The intent of this logical construct is to run a function if a condition is fulfilled in the future.There are 3 states of a promise:

Pending (promise state is on-going)
Resolved (promise was successful)
Rejected (promise was unsuccessful)

Construct a promise

Use a promise constructor to make a new promise. The promise will take as an argument a callback function, to which 2 arguments are passed:

let goodnight = new Promise(function(resolve, reject){

});

The callback function passed to the promise object goodnight is called an executor. resolve and reject are functions that will execute depending on the result.

Example:

let time = new Date().getHours();
let goodnight = new Promise(function(resolve, reject){
  if(time >= 20){
    resolve();
  }else{
    reject();
  }
});

If the condition is fulfilled then the result will be passed to the resolve function which will execute else the reject function which takes an error object as an argument will execute.

The resolve() and reject() functions do not execute immediately. In order to do something with the true/false result received from the callback functions,we use the .then() and .catch() methods

then and catch

goodnight
  .then(function(){
  alert('Goodnight')
})
 .catch(function(){
  alert('Not bedtime')
})

Methods are functions associated with objects. And as promises are objects we can attach the .then() and .catch() methods to our newly created goodnight object. These will "then" do something. If time is >= 8 pm then, the anonymous callback function within then will alert 'Goodnight'. Else 'Not bedtime' will be alerted.

Back to ml5.js

Coming back to ml5.js as mentioned it provides access to various image, video, sound and text machine learning pre-trained models. You can see most of them in the left nav-bar here.

Each one of these pre-trained models comes with:

Parameters: passed during initialization of a pre-trained model.The name of the pre-trained model is required. For example, MobileNet, PoseNet. Other parameters such as a callback function,a video, image etc are optional and depend upon the model you chosen
Properties:These are optional
Methods: These are used to do something with the data for example.detect() will detect a face. The PoseNet model uses an event-handler method to detect new poses .on(pose,callbackFxn)

To use them:

Initialize a pre-trained model:

let posenet = ml5.nameOfMethod(nameOfPreTrainedModel,callbackFxn)

Define callback function:

function callbackFxn(){
console.log('model loaded successfully')
}

Use the model referenced by the variable in #1 to do something:

poseNet.on('pose', function(results){
//do something with the results
pose1 = results;

});

a callback function is passed to the method so that we can do something with the result/data received.

Blog