Read data from Gmail using Node JS

commentme

Unnati Bamania

Posted on April 11, 2021

Read data from Gmail using Node JS

I've struggled a lot with this topic. Finally, after a lot of searches and referring documentation, I'm finally writing this article. I hope it helps everyone. In this article, the code will be written in NodeJS. This app will include 2 parts, the first is creating and setting up a project on google console and the next is the coding part.

1) Creating a project on Google Console

Alt Text
Open https://console.developers.google.com/ website and you need to create a project. On the top left corner, there will be a list of your previous projects or option select a project.
A modal will pop up. Select on the option NEW PROJECT.

  • select -> NEW PROJECT Enter a project name and create the project Alt Text

2) Enable Gmail API

Ensure that you've selected that project, then in the search box search Gmail API. Enable Gmail API.
Alt Text

3) Configure Project

You need credentials but before creating credentials you need to configure credentials. On the left-hand side, you'll find OAuth consent Screen. Click on it.

  • User Type -> External Alt Text
  • Enter app information ie, app name, supporting email, developer contact information. Alt Text
  • Scope page save and continue Alt Text
  • Test User Tab: Click on add user and you can add up to 100 emails. Add your email for testing. Save and continue.
    Alt Text
    Alt Text

  • Finally, after setting up, click on Credentials.

4) Create Credentials

After landing on Credentials, on the top click on CREATE CREDENTIALS. Click on OAuth client ID. Select your application type. As we're using NodeJS, it is a web application. Add URI's as
http://localhost:3000. Create and you'll get your credentials.
Alt Text

Alt Text

Alt Text

Now comes part two, the coding part

5) Code set up:

Go to this page, Node JS quickstart.

  • Copy Paste the sample code and then copy-paste in a file named index.js.
    Alt Text

  • In the folder, where you created this file, the terminal add the command

    npm init

    and initializes

    package.json

  • You need to install some dependencies with the command

    npm i googleapis cheerio mailparser js-base64 open

  • Go to google developers console in your project. Navigate to the credentials part. In OAuth 2.0 Client IDs, you'll find a small download icon, download your credentials file from there and add to your folder where you've created this project. Name this file as

    credentials.json

  • Run your code in your terminal. When you run for the first time, you'll get something like this

Authorize this app by visiting this url: https://accounts.google.com/o/oauth2/v2/auth?access_type=offline&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fgmail.readonly&response_type=code&client_id=479559853488-050lms0ffusprdhh938s954q0s36kg4i.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob
‍Enter the code from that page here:
Enter fullscreen mode Exit fullscreen mode

Click on that URL and enter the code.
Now, in order to be able to manage the labels of the messages, you need to change the initial scope from gmail.readonly to gmail.modify.

const SCOPES = ['https://www.googleapis.com/auth/gmail.modify'];
Enter fullscreen mode Exit fullscreen mode

Delete token.json from your working directory.

One error that some of you might get. The reference code has credentials.installed but it should be credentials.web. When you check the file, credentials.json you'll find everything is inside web object. So, if you get that error just check your creddentials.json file once and replace installed accordingly.

function authorize(credentials, callback) {
  const {client_secret, client_id, redirect_uris} = credentials.installed;
...
}
Enter fullscreen mode Exit fullscreen mode
  • If you're facing any other issue regarding app verification then navigate to verification process and follow them.

6)Final Output:

Then, go to the listLabels function and add the label id in the console log.

functionlistLabels(auth) {  
  ...      
    labels.forEach((label) => {        
      console.log(`- ${label.name} : ${label.id}`);      
    });  
  ...}

Enter fullscreen mode Exit fullscreen mode

Finally, run the application by using the command

npm run server
Enter fullscreen mode Exit fullscreen mode


to authorize the app again and update the access permissions. You should be able to see the label list but now each label with its corresponding id.

Now, to read mail box,

function listMessages(auth, query){
  query = 'unnatibamania8@gmail.com';
  return new Promise((resolve, reject) => {    
    const gmail = google.gmail({version: 'v1', auth});
    gmail.users.messages.list(      
      {        
        userId: 'me',  
        q:query,      
        maxResults:5     
      },            (err, res) => {        
        if (err) {                    reject(err);          
          return;        
        }        
        if (!res.data.messages) {                    resolve([]);          
          return;        
        }                resolve(res.data);  

                         getMail(res.data.messages[0].id, auth);
      }    
    );  
  })
}


Enter fullscreen mode Exit fullscreen mode

Here, the function listMessages takes 2 parameters auth and query, you can call this function

fs.readFile('credentials.json', (err, content) => {
  if (err) return console.log('Error loading client secret file:', err);
  // Authorize a client with credentials, then call the Gmail API.
  authorize(JSON.parse(content), listMessages);

});

Enter fullscreen mode Exit fullscreen mode

after console logging res.data.messages, you'll get threadId and id. Pass the id into getMail function.

function getMail(msgId, auth){
  console.log(msgId)
  const gmail = google.gmail({version: 'v1', auth});
  //This api call will fetch the mailbody.
  gmail.users.messages.get({
      userId:'me',
      id: msgId ,
  }, (err, res) => {
    console.log(res.data.labelIds.INBOX)
      if(!err){
        console.log("no error")
          var body = res.data.payload.parts[0].body.data;

          var htmlBody = base64.decode(body.replace(/-/g, '+').replace(/_/g, '/'));
          console.log(htmlBody)
          var mailparser = new Mailparser();

          mailparser.on("end", (err,res) => {
              console.log("res",res);
          })

          mailparser.on('data', (dat) => {
              if(dat.type === 'text'){
                  const $ = cheerio.load(dat.textAsHtml);
                  var links = [];
                  var modLinks = [];
                  $('a').each(function(i) {
                      links[i] = $(this).attr('href');
                  });

                  //Regular Expression to filter out an array of urls.
                  var pat = /------[0-9]-[0-9][0-9]/;

                  //A new array modLinks is created which stores the urls.
                  modLinks = links.filter(li => {
                      if(li.match(pat) !== null){
                          return true;
                      }
                      else{
                          return false;
                      }
                  });
                  console.log(modLinks);

                  //This function is called to open all links in the array.

              }
          })

          mailparser.write(htmlBody);
          mailparser.end();

      }
  });
}


Enter fullscreen mode Exit fullscreen mode

Final Output

Alt Text
Don't read all the messages all together it might crash the application. Hence, I've read only 1 mail at a time. You can just play around with the code and find a lot of other stuff!
I hope this article helps.

💖 💪 🙅 🚩
commentme
Unnati Bamania

Posted on April 11, 2021

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related