Hello! I recently built a machine learning based web application that can identify a flower (daisy, dandelion, sunflower, rose and tulip for now) using [Google's Teachable Machine](https://teachablemachine.withgoogle.com/) for training a machine learning model and [SashiDo](https://www.sashido.io/en/) for storing images. I thought this was an interesting idea where users can either upload an image or use their webcam to get predicted results and now I'll walk you through it. ##Demo Here's a short demo video showing how the application works: {% youtube 4JGH3JQjizc %} ##Table of Contents * [Teachable Machine Learning](#chapter-1) * [SashiDo](#chapter-2) * [Frontend](#chapter-3) * [WebCam Based Prediction](#chapter-4) * [Uploaded Image Based Prediction](#chapter-5) * [Conclusion](#chapter-6) * [References](#chapter-7) ####Google's Teachable Machine Learning For classifying flowers, the first step is to generate a ML model. [Teachable machine](https://teachablemachine.withgoogle.com/) is a web-based tool that can be used to generate 3 types of models based on the input type, namely Image, Audio and Pose. I created an image project and uploaded images of flowers which were taken from a [kaggle dataset](https://www.kaggle.com/alxmamaev/flowers-recognition). ![TML](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9ia445owel76wau31ouq.png) There are a few advanced settings for epochs,learning rate and batch size, but I felt the default ones were good enough for the task.After training, I exported the model and uploaded it. This stores it in the cloud and gives a shareable public link which can be then used in the project. The next step would be to use to model to perform classification. There are two ways of providing input,we shall go through both of them. ####SashiDo [SashiDo](https://www.sashido.io/en/) is a beautiful backend as a service platform and has a lot of built in functions.In this project, I've used only the Files functionality to store images uploaded by users. I agree that this isn't totally necessary but it is a great way to obtain more samples from the public and build a better dataset. To connect the application with [SashiDo](https://www.sashido.io/en/) copy the code in the getting started page in [SashiDo's](https://www.sashido.io/en/) Dashboard to the javascript file and also add the following script. ``` ``` Next step would be working on the frontend. ####Frontend I've created two buttons to start/stop the webcam and to upload image, an input element for file upload and 3 empty divs to display the webcam input,image input and the output(prediction result). I have used [Bootstrap](https://getbootstrap.com/), so in case you're not familiar with it, the class names basically correspond to various utilities in it. ```

Flower Recognition Application

To identify a Daisy, Rose, Dandelion, Tulip, or Sunflower, either use your web camera and show the flower or upload an image from your device.

USE WEBCAM:

UPLOAD IMAGE:

``` ####WebCam Based Prediction ![Web Based](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ypn3nl7mawi3rjhmomqi.png) The model can be used in our javascript project easily using the [Teachable machine](https://teachablemachine.withgoogle.com/) library for images. To use the library, just add the following scripts at the bottom of the html file. Alternatively, you could also install the library from NPM package installer. ``` ``` The following code helps in toggling the webcam button and declares some variables.The URL constant is set to the model link. ``` let model, webcam, newlabel, canvas, labelContainer, maxPredictions, camera_on = false, image_upload = false; function useWebcam() { camera_on = !camera_on; if (camera_on) { init(); document.getElementById("webcam").innerHTML = "Close Webcam"; } else { stopWebcam(); document.getElementById("webcam").innerHTML = "Start Webcam"; } } async function stopWebcam() { await webcam.stop(); document.getElementById("webcam-container").removeChild(webcam.canvas); labelContainer.removeChild(newlabel); } ``` Now,we can load the model and perform the prediction and display the class having highest probability. ``` // Load the image model and setup the webcam async function init() { const modelURL = URL + "model.json"; const metadataURL = URL + "metadata.json"; // load the model and metadata model = await tmImage.load(modelURL, metadataURL); maxPredictions = model.getTotalClasses(); // Convenience function to setup a webcam const flip = true; // whether to flip the webcam webcam = new tmImage.Webcam(200, 200, flip); // width, height, flip await webcam.setup(); // request access to the webcam await webcam.play(); window.requestAnimationFrame(loop); // append element to the DOM document.getElementById("webcam-container").appendChild(webcam.canvas); newlabel = document.createElement("div"); labelContainer = document.getElementById("label-container"); labelContainer.appendChild(newlabel); } async function loop() { webcam.update(); // update the webcam frame await predict(webcam.canvas); window.requestAnimationFrame(loop); } // run the image through the image model async function predict(input) { // predict can take in an image, video or canvas html element const prediction = await model.predict(input); var highestVal = 0.00; var bestClass = ""; result = document.getElementById("label-container"); for (let i = 0; i < maxPredictions; i++) { var classPrediction = prediction[i].probability.toFixed(2); if (classPrediction > highestVal) { highestVal = classPrediction; bestClass = prediction[i].className; } } if (bestClass == "Daisy" || bestClass == "Dandelion" || bestClass == "Sunflower") { newlabel.className = "alert alert-warning"; } else { newlabel.className = "alert alert-danger"; } newlabel.innerHTML = bestClass; } ``` ####Uploaded Image Based Prediction ![Image Based](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lpmz62vxzdjw4uzi0fjg.png) The second way of providing input is by uploading an image. I've used a little bit of jQuery code to do this. Essentially, once a user selects an image file using the input element on the client side and clicks load,the reference to the file is obtained using a click handler and a new Parse file is created. A Parse file lets us store application files in the cloud that would be too large to store in a object. Next, I created a canvas element to display the saved image and used it to predict the class of the uploaded image. ``` $(document).ready(function () { $("#loadBtn").on("click", async function () { labelContainer = document.getElementById("label-container-cam"); image_upload = !image_upload; if (!image_upload) { labelContainer.removeChild(newlabel); document.getElementById("uploadedImage").removeChild(canvas); } const fileUploadControl = $("#fruitimg")[0]; if (fileUploadControl.files.length > 0) { const modelURL = URL + "model.json"; const metadataURL = URL + "metadata.json"; // load the model and metadata model = await tmImage.load(modelURL, metadataURL); maxPredictions = model.getTotalClasses(); const file = fileUploadControl.files[0]; const name = "photo.jpg"; const parseFile = new Parse.File(name, file); parseFile.save().then(async function () { //The file has been saved to the Parse server img = new Image(224, 224); img.crossOrigin = "Anonymous"; img.addEventListener("load", getPredictions, false); img.src = parseFile.url(); }, function (error) { // The file either could not be read, or could not be saved to Parse. result.innerHTML = "Uploading your image failed!"; }); } else { result.innerHTML = "Try Again!"; } }); }); ``` In the below code, a canvas is created to display the image and prediction is done using the same predict function that was used for webcam. ``` async function getPredictions() { canvas = document.createElement("canvas"); var context = canvas.getContext("2d"); canvas.width = "224"; canvas.height = "224"; context.drawImage(img, 0, 0); document.getElementById("uploadedImage").appendChild(canvas); newlabel = document.createElement("div"); labelContainer = document.getElementById("label-container-cam"); labelContainer.appendChild(newlabel); await predict(canvas); } ``` That's it! The project is now ready to classify flowers. ##Conclusion I learnt a lot by doing this project. I hadn't used [SashiDo](https://www.sashido.io/en/), this is my first time using it, but it made the backend process really simple. Also, I had to learn about jquery, as this is my first time writing code in jquery. [Google's Teachable machine](https://teachablemachine.withgoogle.com/) helped a lot in making the machine learning model, it made the overall process very smooth and efficient. I hope you enjoyed reading this. This is a pretty simple project, so if you have some time and are interested, go ahead and try building it yourself! [GitHub Repo](https://github.com/ygarg704/Flora) [Project Link](https://yashgarg.xyz/Flora/) ###References SashiDo Starter Guide [Part 1](https://blog.sashido.io/sashidos-getting-started-guide/) SashiDo Starter Guide [Part 2](https://blog.sashido.io/sashidos-getting-started-guide-part-2/) The Awesome Teachable Machine Learning [List](https://github.com/SashiDo/awesome-teachable-machine) Teachable Machine [Node](https://github.com/SashiDo/teachablemachine-node) Parse [SDK](https://github.com/parse-community/Parse-SDK-JS) Parse [File](https://docs.parseplatform.org/js/guide/#creating-a-parsefile) [Teachable Machine](https://teachablemachine.withgoogle.com/)

ygarg704

How to Build a Flower Recognition App with Teachable Machine and SashiDo

Join Our Newsletter. No Spam, Only the good stuff.