Building an Image-Based Search app with Google Lens API

Let's learn how to build a simple Google Lens clone using Google Lens API. We'll learn how to upload the image and use it as a search method before returning all the visual matches.

Text is not the only way to search for something. With Google Lens, you can search almost anything through an image instead.

Using Google Lens, you can do things like:

Finding where an outfit is sold on the internet.
Translate any text in real time.
Identifying objects, plants, or animals.
Even help with your homework!

While Google doesn't have an official API for Google Lens, luckily, SerpAPI provides us with a Google Lens API alternative. This API allows us to scrape results from the Google Lens page when performing an image search. You can get a knowledge graph, visual matches, text, and other data as well.

In this post, we'll use Node.js, but feel free to use another programming language. Here is the "framework":

- Create an API endpoint where the user can upload an image

- Upload the image to any hosting image and return the URL

- Pass this image's URL to the Google Lens API parameter

- Provide the relevant information

Upload image functionality

Let's start building our image-uploader API. Since I'm using Javascript (Node.js), I'll use these packages for help:

- Formidable - Flexible, fast, and streaming parser for multipart form data. It's used to accept our image file.

- Axios - We need to perform an external HTTP request to upload the image. We can use Axios or the built-in HTTP Node library.

- IMGBB - ImgBB is a free image hosting provider with an API. We'll upload our images here. So grab your free imgbb API key here (make sure to sign up first)! You can use other image hosting providers like Imgur as well.

You can use the same codebase for using Google Reverse Image API

Step 1: Preparing a new project

Let's create a new folder and initialize the NPM here.

mkdir google-lens-api
cd google-lens-api
npm init -y

// Create a new file
touch main.js

Install all the packages that we need.

npm install axios formidable --save

Step 2: Writing the NodeJS Code

Here's the code to upload an image with NodeJS:

const http = require('http');
const {formidable, errors} = require('formidable');
const fs = require('fs');
const axios = require('axios');

const IMGBB_API_KEY = 'YOUR-IMGBB-KEY'; 

const server = http.createServer(async (req, res) => {
  if (req.url === '/upload' && req.method.toLowerCase() === 'post') {
     // parse a file upload
     const form = formidable({});
     let fields;
     let files;
     try {
         [fields, files] = await form.parse(req);

        //  if empty files
        if (Object.keys(files).length === 0) {
            throw new Error('No files were uploaded.');
        }

        const image = files.image[0];
        // Read the file data
        const fileData = fs.readFileSync(image.filepath);

        // Convert file data to a base64 encoded string
        const base64Image = new Buffer.from(fileData).toString('base64');

         // Prepare the payload to imgBB
         const formData = new URLSearchParams();
         formData.append('image', base64Image);

          // Send the request to imgBB
          const imgBBResponse = await axios.post(`https://api.imgbb.com/1/upload?key=${IMGBB_API_KEY}`, formData, {
              headers: {
                  'Content-Type': 'application/x-www-form-urlencoded'
              }
          });

         // Send the response from imgBB
         res.writeHead(200, { 'Content-Type': 'application/json' });
         res.end(JSON.stringify(imgBBResponse.data));
         return

     } catch (err) {
         // example to check for a very specific error
         if (err.code === errors.maxFieldsExceeded) {

         }
         console.error(err);
         res.writeHead(err.httpCode || 400, { 'Content-Type': 'text/plain' });
         res.end(String(err));
         return;
     }
  } else {
    // Handle 404 - Not Found
    res.writeHead(404, { 'Content-Type': 'text/plain' });
    res.end('404 Not Found');
  }
});

server.listen(3000, () => {
  console.log('Server listening on http://localhost:3000/');
});

Code explanation:

- We're using built-in Node HTTP server. We use /upload as a path to our image upload API.

- We grab the uploaded file using formidable.

- Turn the image file into base64 image.

- Using axios, upload this base64 image to IMGBB Api

Make sure to run your Nodejs server

node main.js

Step 3: Test the API in Postman

Let's test this API! Here's what my Postman looks like:

Our target URL is localhost:3000/upload using the POST method
Passing parameter on Body > Form-Data
Make sure to use the same key name on your backend. In this case, it is image
Choose file format instead of text
Choose your image file on the value section.

Try to send this request. Now we get the uploaded image URL from imgBB's response at data > url . We can then use this as a parameter later.

Using Google Lens API

Now, it's time to use the Google Lens API. The only required parameter is url , which represents the URL of the image we want to search for. We've got this covered, so let's continue with the search API itself.

Make sure to register an account at SerpApi to get the API key. You'll get 100 free search credit each month,

Step 1: Install SerpApi - Javascript integration

We have SerpApi - Javascript integration here. Install it with:

npm install serpapi

Step 2: Send an API request to SerpApi

Grab your SerpApi Key from our dashboard and pass this in the search parameter.

const { getJson } = require("serpapi");

// ... all the previous code
// const imgBBResponse = await ...

const imageUrl = imgBBResponse.data.data.url;
const response = await getJson({
  engine: "google_lens",
  api_key: "YOUR_SERPAPI_KEY", // From https://serpapi.com/manage-api-key
  url: imageUrl,
  location: "Austin, Texas",
})

// Send the response from imgBB
res.writeHead(200, { 'Content-Type': 'application/json' });
res.end(JSON.stringify(response, null, 2));

Now you can see the Google lens response in a nice structured JSON format

Depending on our needs, we can return only the specific parts like the knowledge graph, shopping results, text results, and visual matches. You can loop the visual_matches to display the image results from the thumbnail attribute.

A client app example

Let's see how to build a website to upload the image and display the results using HTML.

But first, we need to make sure that our backend code is ready for external access by allowing CORS.

In your main.js file, add this code:

const server = http.createServer(async (req, res) => {
   // Set CORS headers
   res.setHeader('Access-Control-Allow-Origin', '*'); // to allow any origin
   res.setHeader('Access-Control-Allow-Methods', 'OPTIONS, GET, POST, PUT, PATCH, DELETE'); // methods you want to allow
   res.setHeader('Access-Control-Allow-Headers', 'Content-Type, Authorization'); // headers you want to allow

   // Handle OPTIONS method (pre-flight request)
   if (req.method === 'OPTIONS') {
     res.writeHead(204); // No Content
     res.end();
     return;
   }
   ....

Here is our basic HTML code to send the request to API when the user chooses an image.

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Document</title>
</head>
<body>
    <div id="app">
        <input type="file" id="fileUploadBtn">
        <div id="resultBox"></div>
    </div>

    <script>
        // Fecth post to http://localhost:3000/upload

        const fileUploadBtn = document.querySelector('#fileUploadBtn');
        fileUploadBtn.addEventListener('change', (e) => {
            const file = e.target.files[0];
            const formData = new FormData();
            formData.append('image', file);

            fetch('http://localhost:3000/upload', {
                method: 'POST',
                body: formData
            })
            .then(res => res.json())
            .then(data => console.log(data))
            .catch(err => console.log(err));
        });

    </script>
</body>
</html>

Instead of just logging the result, let's add the results visually in our code

 <script>
        // Fecth post to http://localhost:3000/upload

        const fileUploadBtn = document.querySelector('#fileUploadBtn');
        fileUploadBtn.addEventListener('change', (e) => {
            const file = e.target.files[0];
            const formData = new FormData();
            formData.append('image', file);

            fetch('http://localhost:3000/upload', {
                method: 'POST',
                body: formData
            })
            .then(res => res.json())
            .then(data => {
                const resultBox = document.querySelector('#resultBox');
                const visualMatches = data.visual_matches

                visualMatches.forEach(element => {
                    const box = `
                        <div class="card">
                            <img src="${element.thumbnail}" alt="">
                            <div class="content">
                                <a href="${element.link}">${element.title}</a>
                            </div>
                        </div>
                    `

                    resultBox.innerHTML += box;
                });

            })
            .catch(err => console.log(err));
        });

    </script>

Feel free to use your own styling. Here is mine:


    <style>
        #resultBox {
            display: flex;
            flex-wrap: wrap;
        }

        .card {
            width: 200px;
            height: 300px;
            margin: 10px;
            border: 1px solid #ccc;
            border-radius: 5px;
            overflow: hidden;
            text-align: center;
        }
    </style>

Here is what our Google Lens clone looks like; it provides the image thumbnails, complete with the title and linked resources.

AWS S3 as an image Hosting provider and the alternative

We have had our users have problems when using Amazon S3 as a hosting image provider. Google can't access those images on S3, and due to that, you probably get errors.

Our recommendation is to use a public image hosting provider like imgb, Imgur, Uploadcare, Cloudinary, etc.

Alternatively, you rent a VPS or a shared hosting environment to upload your images and keep them accessible via HTTP requests behind a NginX or Apache Web Server.

Here is an example of serving static files with NGINX

server {
    root /www/data;

    location / {
    }

    location /images/ {
    }
}

Here, NGINX searches for a URI that starts within/images/ the /www/data/images/ directory in the file system. Resource: Nginx Documentation.

Search images programmatically without an upload button

If you want to search by multiple images and store the results somewhere, like in CSV format. You might not need to implement a GUI for the uploading part.

You can drop the "client app" HTML part, as well as the backend code where we use formidable .

Let's say we have images folder on the same root as our main.js file. Here is how to loop this file

const fs = require('fs');
const path = require('path');

const directoryPath = path.join(__dirname, 'images');
fs.readdir(directoryPath, function (err, files) {
    if (err) {
        return console.log('Unable to scan directory: ' + err);
    }
    files.forEach(function (file) {
        console.log(file);
        // Process each file
    });
});

Now, you can perform the same logic to convert each of the files to a base64 image before uploading it to an image hosting provider.

Hope it helps! Thank you for reading this post!

Blog