Real-Time Sentiment Analysis on Messages

Symbl.ai's API for sentiment analyzes messages in real-time, returning their polarity with a suggestion. If the polarity is below .5, the suggestion is negative. If above, the suggestion is positive. The suggestion, however, may be modified. If, for instance, you want to program a switch to toggle suggestions, the switch statement could return results for granulated polarities, since Symbl.ai's API for sentiment is fully programmable.

In the following blog you create a web app with which to map sentiments directly to message IDs in real-time with Symbl.ai's sentiment analysis API over a WebSocket in JavaScript running locally on a Python server. The result is a table of message IDs with sentiments.

A table of message IDs with sentiments may not seem like a whole lot but for a fully programmable API, there can be nothing better than generic functionality. So it is important to disclaim at the outset that the web app wastes absolutely no time on a user interface beyond what is required to demonstrate the generic functionality of the API.

Create the Web APP

In the web app, create the following files: index.html with a source folder containing an index.js file together with a style.css file.

In the style.css file, add the following lines:

body {
  font-family: sans-serif;
}

In the index.html, add the following lines:

<!DOCTYPE html>
<html>

<head>
    <title>Parcel Sandbox</title>
    <meta charset="UTF-8" />

    <script src="src/index.js">
    </script>
</head>

<body>
    <div id="app"></div>

    <button type="button" onclick="openSocket()">Click Me!</button>

    <div id="table-parent">
    </div>
</body>

</html>

The index.html achieves two goals. It creates a button for triggering the openSocket() method. The second is a table. In the openSocket() you program the JavaScript WebSocket for Symbl.ai's WebSocket for its platform. In the table you program a log, logging the message ID with its polarity score. You do not return to these two files later.

Sentiment Analysis: WebSocket in JavaScript & Events

The first step to sentiment analysis on messages with Symbl.ai is to establish a WebSocket. The second step is make a call to the sentiment API as the WebSocket's handler for events handles events the speaker makes.

To establish a WebSocket connection, the first step is to sign up for a free account at Symbl.ai. Register for an account at Symbl (i.e., https://platform.symbl.ai/). Grab both your appId and your appSecret. With both of those you authenticate either with a cURL command or with Postman so that you receive your x-api-key. Here is an example with cURL:

curl -k -X POST "https://api.symbl.ai/oauth2/token:generate" \
     -H "accept: application/json" \
     -H "Content-Type: application/json" \
     -d "{ \"type\": \"application\", \"appId\": \"<appId>\", \"appSecret\": \"<appSecret>\"}"

After signing you, you receive free credits with which to make API calls. To make API calls on a WebSocket connection with Symbl.ai, create four const in your index.js file:

 const accessToken = "";
  const uniqueMeetingId = btoa("devrelations@symbl.ai");
  const symblEndpoint = `wss://api.symbl.ai/v1/realtime/insights/${uniqueMeetingId}?access_token=${accessToken}`;
  const ws = new WebSocket(symblEndpoint);

The WebSocket connection requires both an unique meeting ID, as well as an accessToken (i.e., x-api-key you generate in cURl or Postman with a request containing an appId together an appSecret).

After configuring the endpoint for a WebSocket with Symbl.ai, add the following methods for handling events, 1) ws.onmessage, 2) ws.onerror, 3) ws.onclose.

// Fired when a message is received from the WebSocket server
ws.onmessage = (event) => {
  console.log(event);
};
// Fired when the WebSocket closes unexpectedly due to an error or lost connetion
ws.onerror  = (err) => {
  console.error(err);
};
// Fired when the WebSocket connection has been closed
ws.onclose = (event) => {
  console.info('Connection to websocket closed');
};

After these methods are created, create a method called onopen in the following way:

// Fired when the connection succeeds.
ws.onopen = (event) => {
  ws.send(JSON.stringify({
    type: 'start_request',
    meetingTitle: 'Establish a WebSocket Connection', // Conversation name
    insightTypes: ['question', 'action_item'], // Will enable insight generation
    config: {
      confidenceThreshold: 0.5,
      languageCode: 'en-US',
      speechRecognition: {
        encoding: 'LINEAR16',
        sampleRateHertz: 44100,
      }
    },
    speaker: {
      userId: 'devrelations@symbl.ai',
      name: 'Developer Relations',
    }
  }));

The onopen method contains many parts, the most important of which is speechRecognition where encoding is set to LINEAR16 while the hertz is set to 44100. To read more about best practices for streaming audio integrations, checkout the following blog: https://symbl.ai/best-practices-for-audio-integrations-with-symbl/.

The last but not least step is to configure the WebSocket to access the client's device (i.e., microphone).

const stream = await navigator.mediaDevices.getUserMedia({ audio: true, video: false });
const handleSuccess = (stream) => {
  const AudioContext = window.AudioContext;
  const context = new AudioContext();
  const source = context.createMediaStreamSource(stream);
  const processor = context.createScriptProcessor(1024, 1, 1);
  const gainNode = context.createGain();
  source.connect(gainNode);
  gainNode.connect(processor);
  processor.connect(context.destination);
  processor.onaudioprocess = (e) => {
    // convert to 16-bit payload
    const inputData = e.inputBuffer.getChannelData(0) || new Float32Array(this.bufferSize);
    const targetBuffer = new Int16Array(inputData.length);
    for (let index = inputData.length; index > 0; index--) {
        targetBuffer[index] = 32767 * Math.min(1, inputData[index]);
    }
    // Send to websocket
    if (ws.readyState === WebSocket.OPEN) {
      ws.send(targetBuffer.buffer);
    }
  };
};
handleSuccess(stream);

A detailed examination of a WebSocket's access to client's device is beyond the scope of the current blog, since our focus is on real-time sentiment analysis on messages.

The following is the full code for establishing the WebSocket connect:

const uniqueMeetingId = btoa('email@address.com');
const accessToken = '';
const symblEndpoint = `wss://api.symbl.ai/v1/realtime/insights/${uniqueMeetingId}?access_token=${accessToken}`;
const ws = new WebSocket(symblEndpoint);
// Fired when a message is received from the WebSocket server
ws.onmessage = (event) => {
  console.log(event);
};
// Fired when the WebSocket closes unexpectedly due to an error or lost connetion
ws.onerror  = (err) => {
  console.error(err);
};
// Fired when the WebSocket connection has been closed
ws.onclose = (event) => {
  console.info('Connection to websocket closed');
};
// Fired when the connection succeeds.
ws.onopen = (event) => {
  ws.send(JSON.stringify({
    type: 'start_request',
    meetingTitle: 'Websockets How-to', // Conversation name
    insightTypes: ['question', 'action_item'], // Will enable insight generation
    config: {
      confidenceThreshold: 0.5,
      languageCode: 'en-US',
      speechRecognition: {
        encoding: 'LINEAR16',
        sampleRateHertz: 44100,
      }
    },
    speaker: {
      userId: 'example@symbl.ai',
      name: 'Example Sample',
    }
  }));
};
const stream = await navigator.mediaDevices.getUserMedia({ audio: true, video: false });
const handleSuccess = (stream) => {
  const AudioContext = window.AudioContext;
  const context = new AudioContext();
  const source = context.createMediaStreamSource(stream);
  const processor = context.createScriptProcessor(1024, 1, 1);
  const gainNode = context.createGain();
  source.connect(gainNode);
  gainNode.connect(processor);
  processor.connect(context.destination);
  processor.onaudioprocess = (e) => {
    // convert to 16-bit payload
    const inputData = e.inputBuffer.getChannelData(0) || new Float32Array(this.bufferSize);
    const targetBuffer = new Int16Array(inputData.length);
    for (let index = inputData.length; index > 0; index--) {
        targetBuffer[index] = 32767 * Math.min(1, inputData[index]);
    }
    // Send to websocket
    if (ws.readyState === WebSocket.OPEN) {
      ws.send(targetBuffer.buffer);
    }
  };
};
handleSuccess(stream);

Run the code directly in your browser's console without any reference to the web app. If you were able to log messages in the console, you successfully established a WebSocket connection. The next step is to configure onmessage to log the polarity score on those messages for realtime sentiment analysis.

Symbl.ai's Real-Time Sentiment Analysis API

The next step is to configure onmessage to log the polarity score on those messages for realtime sentiment analysis. The first step to analyze sentiments is to log the message IDs. You reconfigure onmessage to log message IDs.

Logging Message IDs

Our objective now is to make a call to the following API endpoint:

https://api.symbl.ai/v1/conversations/${conversationId}/messages?sentiment=true

You note that passing ?sentiment=true into the API is the query parameter for a request to return messages with values for polarities on message IDs. Since the API endpoint requires no more than a ${conversationId}, the first step is to assign the conversationId to a constant.

  // You can find the conversationId in event.message.data.conversationId;
  const data = JSON.parse(event.data);
  if (data.type === 'message' && data.message.hasOwnProperty('data')) {
    console.log('conversationId', data.message.data.conversationId);
    const conversationId = data.message.data.conversationId;
console.log('onmessage event', event);
    // You can log sentiments on messages from data.message.data.conversationId

With the conversationId the next step is to configure an HTTP request to make a call to the API for sentiment analysis every time that the WebSocket logs an event. To configure an HTTP request to make a call to the API, make the call by configuring the headers as well as the authorization.

   const request = new XMLHttpRequest();
    request.responseType = "text";
    const sentimentEndpoint = `https://api.symbl.ai/v1/conversations/${conversationId}/messages?sentiment=true`;
    request.open("GET", sentimentEndpoint)
    request.setRequestHeader('Authorization', `Bearer ${accessToken}`);
    request.setRequestHeader('Content-Type', 'application/json');
    request.onreadystatechange=(e)=> {
      console.log(request.responseText)
    }
    request.send()
  }

With the request configured, the API endpoint makes a call every time the WebSocket handles an event that a speaker triggers. If you want, run the code in your console. It logs polarity values for message IDs. However, these logs do not map one to the other. The following is the full code for establishing the WebSocket connect:

// Fired when a message is received from the WebSocket server
ws.onmessage = (event) => {
  // You can find the conversationId in event.message.data.conversationId;
  const data = JSON.parse(event.data);
  if (data.type === 'message' && data.message.hasOwnProperty('data')) {
    console.log('conversationId', data.message.data.conversationId);
    const conversationId = data.message.data.conversationId;
    console.log('onmessage event', event);
    // You can log sentiments on messages from data.message.data.conversationId 
    const request = new XMLHttpRequest();
    request.responseType = "text";
    const sentimentEndpoint = `https://api.symbl.ai/v1/conversations/${conversationId}/messages?sentiment=true`;
    request.open("GET", sentimentEndpoint)
    request.setRequestHeader('Authorization', `Bearer ${accessToken}`);
    request.setRequestHeader('Content-Type', 'application/json');
    request.onreadystatechange=(e)=> {
      console.log(request.responseText)
    }
    request.send()
  }
};

Refactoring the Code

It is time to refactor our code to conform to the web app built earlier, as well as log sentiments on messages. In the web app an element called table-parent is identified as <div id="table-parent">. The call to the API endpoint feeds both the message IDs together with the scores for polarity directly into the table-parent in real-time.

Refactoring the API call

Refactor the API call in the following way:

if (conversationId) {
        // You can log sentiments on messages from data.message.data.conversationId
        const sentimentEndpoint = `https://api.symbl.ai/v1/conversations/${conversationId}/messages?sentiment=true`;
        const response = await fetch(sentimentEndpoint, {
          method: 'GET',
          mode: 'cors',
          cache: 'no-cache',
          headers: {
            'Content-Type': 'application/json',
            'Authorization': `Bearer ${accessToken}`
          }
        });

Configuring the Table

const resp = await response.json();
        if (response.ok) {
          let rows = "";
          for (let message of resp.messages) {
            if (cacheTable.indexOf(message.id) === -1) {
              console.log('Polarity: ', message.sentiment.polarity.score);
            }
            rows += `
              <tr>
                <td>${message.id}</td>
                <td>${message.sentiment.polarity.score}</td>
              </tr>
            `
            cacheTable.push(message.id);
          }
          let tableHtml = `
            <table>
              <thead>
                <tr>
                  <th>ID</th>
                  <th>Polarity</th>
                </tr>
              </thead>
              <tbody>
              ${rows}
              </tbody>
            </table>
          `;
          debugger;
          document.querySelector("#table-parent").innerHTML = tableHtml;
        }
      }
    }

The parent-table updates when a user speaks or listens.

Refactored Code

The following is the fully refactored code for mapping polarity values to message IDs in a table over a WebSocket connection in JavaScript with Symbl.ai's real-time sentiment analysis API:

ws.onmessage = async (event) => {
    // You can find the conversationId in event.message.data.conversationId;
    const data = JSON.parse(event.data);
    if (data.type === 'message' && data.message.hasOwnProperty('data')) {
      console.log('conversationId', data.message.data.conversationId);
      conversationId = data.message.data.conversationId;
      console.log('onmessage event', event);
    }
    if (data.type === 'message_response') {
      for (let message of data.messages) {
        console.log('Transcript (more accurate): ', message.payload.content);
      }
      if (conversationId) {
        // You can log sentiments on messages from data.message.data.conversationId
        const sentimentEndpoint = `https://api.symbl.ai/v1/conversations/${conversationId}/messages?sentiment=true`;
        const response = await fetch(sentimentEndpoint, {
          method: 'GET',
          mode: 'cors',
          cache: 'no-cache',
          headers: {
            'Content-Type': 'application/json',
            'Authorization': `Bearer ${accessToken}`
          }
        });
        const resp = await response.json();
        if (response.ok) {
          let rows = "";
          for (let message of resp.messages) {
            if (cacheTable.indexOf(message.id) === -1) {
              console.log('Polarity: ', message.sentiment.polarity.score);
            }
            rows += `
              <tr>
                <td>${message.id}</td>
                <td>${message.sentiment.polarity.score}</td>
              </tr>
            `
            cacheTable.push(message.id);
          }
          let tableHtml = `
            <table>
              <thead>
                <tr>
                  <th>ID</th>
                  <th>Polarity</th>
                </tr>
              </thead>
              <tbody>
              ${rows}
              </tbody>
            </table>
          `;
          debugger;
          document.querySelector("#table-parent").innerHTML = tableHtml;
        }
      }
    }
    if (data.type === 'topic_response') {
      for (let topic of data.topics) {
        console.log('Topic detected: ', topic.phrases)
      }
    }
    if (data.type === 'insight_response') {
      for (let insight of data.insights) {
        console.log('Insight detected: ', insight.payload.content);
      }
    }
    if (data.type === 'message' && data.message.hasOwnProperty('punctuated')) {
      console.log('Live transcript: ', data.message.punctuated.transcript);
    }
    // console.log(`Response type: ${data.type}. Object: `, data);
  };

To run the code locally, you have to avoid CORS. At the same time, you need to create an HTTP server in Python. Run the following line of code:

python3 -m http.server 8000

Python3 enables an http server to run locally on your host. With the application running, hit the click button. After hitting the click button, you should see message IDs mapped to values for polarity in real-time. If you run Command + Option + J, the following logs appear in your console.

API's Rigor with Symmetrical Augmentations

Symbl.ai's API is rigorous. It provides for sentiments in a way that other APIs do not. Symbl.ai's sentiment analysis, for instance, provides symmetrical augmentations for adverbial enhancements. If, for instance, you check the sentiment for "It is good", the score is .8. If you check the sentiment for "It is really good", the score is .9. What is true for positivity is true for negativity.

Conclusion

If you were able to successfully integrate Symbl’s API directly into JavaScript’s own software for enabling real-time conversations so that your transcribed a conversation live from the browser, congratulations!
If you look closely at the data, the conversationId may be applied to new API calls to access AI insights for action items, topics, etc… You can hit these API end points with cURL commands, Postman, or take a look at the section on Further Developer down below for ideas.

Community

Stuck? Free feel to ask us any questions on our Slack Channel or send us an email at devrelations@symbl.ai

Blog