About this Series

In this series, I'll be exploring how to develop an event-driven web applications in Javascript. We'll use Vue on the frontend and FeathersJS on the backend to accomplish this. If you're interested in learning more about developing real-time and streaming data applications, follow this series. This will include developing chat-based apps, streaming videos or audio apps, device-driven apps, such as those in the Internet-of-Things space, and much more.

This series assumes familiarity with Javascript. It also assumes familiarity with basic Vue and Node.js concepts, such as setting up a simple project in Vue/Node. I'll do my best to break down more complex concepts. If anything is unclear, please leave a comment so the article can be updated for clarity.

What is Realtime and Streaming Data?

Real-time data is data that is delivered immediately after collection. With ongoing improvements in hardware and computing power, it has become increasingly common for businesses to provide real-time analysis of data to identify potential issues or opportunities. Data collected can be transformed "on-the-fly" and presented to decision-makers instantly.

Traditionally, this was seen with devices and Geographical Information Systems (GIS) which would frequently emit sensor and/or location data.

With the increasing use of AI and data warehousing techniques, it's fairly common now to see real-time processed data. For high-volume applications, it's increasingly important to be able to update sites in real-time as the state of the system changes.

Real-time Events and Data Scenarios

Applications of real-time data will only continue to increase with time. Here are a few common ones to think about:

Health and Fitness Devices

As technology continues to improve, the advent of devices providing instant feedback will continue to increase in order to optimize the care and service doctors can provide. Medical equipment full of sensors will often need to transmit the information instantly in order to provide the doctor and patient the information needed to make informed decisions. In the past, X-rays would take days to process and develop. Now it's available within minutes. Other similar diagnostic procedures are increasingly becoming available to provide near real-time feedback for doctors to make decisions on a patient's condition.

Similarly, fitness trackers can transmit data such as heart rate and record your activity as you exercise or sleep. It can alert you when you've hit your daily step goals or warn you that you're working too hard. These devices all rely on real-time updates to keep the user informed as an event occurs.

E-Commerce & Scheduling

Managing inventory is important for customer satisfaction. Inventory is also finite, so when a user makes a purchase, the item is typically deducted from inventory. This generally works fine on low volume sites where a single user may only make one purchase for a single item at any given time. But what happens when multiple users try to purchase the same item at the same time?

Only one person will be able to complete the purchase. The other orders will need to be canceled once it's been discovered that the product is no longer available. This can lead to a terrible customer experience if the time taken to handle this exceeds a customer's patience and expectations.

Through real-time event updates, customers can be informed the product has been purchased and the item can be removed from their shopping cart before the purchase can be completed. This would help better manage customer expectations. The same can be applied to booking or scheduling applications.

Operational Awareness

Sometimes, monitoring data in real-time is important for business operations. This is generally true for any kind of heuristics or diagnostic platform for computing systems. For example, in preventing and mitigating cyberattacks, it's often necessary to track the flow of traffic entering a network.

The sooner an attack is discovered, the more likely a business can recover from the attack or defend against an attack. In such cases, real-time updates are important to accurately display the current situation.

Working with Realtime Data

The most common way to receive real-time updates on a website is through a real-time transport such as a socket. Sockets maintain an open channel with the server, allowing data and event notifications to pass through.

Socket.io is a popular library to support such connections. FeathersJS supports this out-of-the-box and provides additional scaffolding features for building a robust backend to support real-time applications.

Getting Started with FeathersJS

Getting started with Feathers is easy. I'll briefly go over how to create your own project so you can start using it. Afterward, I'll be using a pre-built project template to demonstrate different use cases. You can either create your own project or follow along using the same template.

Feathers Command Line Interface (CLI)

Feathers provides a CLI to enable you to quickly generate a new application. Globally install the Feathers CLI to generate an app:

npm install @feathersjs/cli -g

Create a folder for your project.

mkdir feathers-realtime
cd feathers-realtime/
feathers generate app

The Feathers CLI will prompt you with questions to help you configure your project, including auth, test frameworks, and data source providers. Adjust these based on your preferences. Make sure to select socket.io for your project when asked about APIs. Once complete, the CLI will automatically generate the project directory structure and files.

To learn more about the generated files, visit the docs.

Project Template

To start with a bit more functionality, I'm going to work from existing project templates within the FeathersJS community and build off these examples.

For the frontend, we'll use the feathers-vuex-chat frontend as a starting point which leverages the feathers-vuex library:

yarn install

yarn serve

yarn build

yarn lint

For the backend, we'll use the feathers-chat backend as a starting point:

cd feathers-chat-ts
npm install

npm run compile
npm run migrate

npm start

npm run dev

The combined repo for this post can be found here:

Real-time Transports

As mentioned above, Feathers supports Socket.io as a real-time transport. It also supports Primus, which is a wrapper for real-time frameworks, making it possible to adapt Feathers to existing real-time frameworks used by other parts of the business.

Hello World - Pushing Messages to the Frontend

To kick off this project, I'm going to mock up some data on the backend to demonstrate real-time updates on the front end. We'll create a simple dashboard with different charts to display real-time data. We'll dive into more use-cases in the next series of posts.

Running the Project

This template uses vue on the frontend. To run the development server, use yarn serve within the feathers-chat-vuex directory. This will start on port 8080 by default. Navigate to your browser, http://localhost:8080/ to view the web app.

This project uses FeatherJS on the backend. To run the development server, use npm run dev. This will start on http://localhost:3030 by default.

The frontend should already be configured to connect to the backend on port 3030 through the /src/feathers-client.js setup.

Mocking the Data

To keep this first post simple, I'll mock up data to be sent by the Feathers backend at regular intervals. We'll use event listeners to detect when a user connects to the server and begin the data push once a user connects.

In channels.js, every time a connection is established with the server, it begins sending data every 5 seconds. This data is randomly generated using a helper function, getRandomInt. It provides data variables that I'll use to update the charts.

For a more realistic use-case, this data could be provided by a service or other data source (see below for a custom service implementation, which is a better approach). Consider sending logs which may provide a constant stream of log data. Or maybe you want to send binary data to display to the user, like an audio clip or video generated by another user.

  let logins = [
    getRandomInt(50, 70),
    getRandomInt(50, 70),
    getRandomInt(50, 70),
    getRandomInt(50, 70),
    getRandomInt(50, 70),
    getRandomInt(50, 70),
    getRandomInt(50, 70),
  ];

  let interval;
  app.on("connection", (connection) => {
    // On a new real-time connection, add it to the anonymous channel
    app.channel("anonymous").join(connection);
    // create 5 second interval to emit "dataAvailable" event with data payload
    interval = setInterval(() => {
      console.log("Sending new data");
    // remove one value, add a new one
      logins.shift();
      logins.push(getRandomInt(50, 70));
    // send the data through the 'dataAvailable' event
      app.io.emit("dataAvailable", {
        messageCount: getRandomInt(1000, 10000) + getRandomInt(0, 100),
        activeChatRooms: getRandomInt(5, 100),
        recentLogins: logins,
        openTickets: getRandomInt(0, 100),
        closedTickets: getRandomInt(0, 100),
        unassignedTickets: getRandomInt(0, 100),
      });
    }, 5000);
  });

  app.on("disconnect", (connection) => {
    clearInterval(interval);
  });

When you run npm run dev to start the server, the server should start transmitting data once a user connects.

Sockets Handling in the Frontend

Feathers provides a wrapper for the socket.io client that works seamlessly with a Feathers backend. Feathers-vuex integrates this library and also provides real-time socket event support within the vuex store. To get started, add the following libraries if not already in your project:

yarn add @feathersjs/feathers @feathersjs/socketio-client @feathersjs/authentication-client socket.io-client @vue/composition-api feathers-vuex feathers-hooks-common

These packages have already been added to the project template. @feathersjs/feathers, @feathersjs/socketio-client, @feathersjs/authentication-client, and socket.io-client provide the connection framework to communicate with your Feathers server through the socket.io real-time transport. The remaining libraries provide support for Vue/Vuex on the frontend.

By default, the feathers-vuex library defaults to real-time connections (the alternative being REST API calls, which you can also configure).

If this is your first time using Feathers-Vuex, I would recommend reviewing the docs, which documents the setup process and key concepts, such as the Auth Plugin, the Service Plugin, and Data Modeling. While this series will cover concepts relevant to the use-cases described, it will not be possible to cover everything.

Dashboard

To demonstrate the continuous stream of data, I'll be creating a simple dashboard filled with charts.

Creating a Dashboard View

// /src/views/Dashboard.vue
<template>
  <main class="home container">
    <div class="row text-center">
      <h1>Dashboard</h1>
    </div>
    <div class="row">
      <div class="col-6">
        <h3>Messages Sent</h3>
        <BarChart :chart-data="barchartdata" :options="options" />
      </div>
      <div class="col-6">
        <h3>Active Chat Rooms</h3>
        <BarChart :chart-data="barchartdata2" :options="options" />
      </div>
    </div>
    <div class="row">
      <h3>Recent Logins</h3>
      <LineChart :chart-data="linechartdata" :options="options" />
    </div>
    <div class="row">
      <h3>Current Tasks</h3>
      <DonutChart :chart-data="donutchartdata" :options="doptions" />
    </div>
    <div class="row">
      <h3>DEBUG</h3>
      {{ serverMessage }}
    </div>
  </main>
</template>

You may notice chart components added to this dashboard view. We'll create these down below.

Adding the View to the Routes

const routes = [
... 
  { path: '/chat', name: 'Chat', component: Chat },
  { path: '/dashboard', name: 'Dashboard', component: Dashboard },
...
];

Adding a Dashboard Link to the Chat View

<div class="title-wrapper block center-element">
  <img
     class="logo"
     src="http://feathersjs.com/img/feathers-logo-wide.png"
     alt="Feathers Logo"
  />
  <span class="title">Chat</span>
</div>
<router-link class="float-right link" to="/dashboard">
    Dashboard
</router-link>

Displaying the Data

To visualize the flow of data, we'll use charts to display data updates. I'm going to use the vue wrapper library vue-chartjs for Charts.js, which provides a simple yet customizable charting library.

yarn add vue-chartjs chart.js

Creating Chart Components

vue-chartjs makes it easy to add charts as a chart component within a single vue component file. View the documentation to learn more about how it can be used within a vue app.

Here is an example of the bar chart component.

// /src/components/BarChart.vue
<script>
import { Bar, mixins } from 'vue-chartjs';
const { reactiveProp } = mixins;

export default {
  extends: Bar,
  mixins: [reactiveProp],
  props: ['chartData', 'options'],
  mounted() {
    this.renderChart(this.chartData, this.options);
  },
};
</script>

Make sure you include the mixins and reactiveProp. The reactiveProp mixin adds a watcher to the chartData variable, enabling updates as data changes.

Listening to Events

To create an event listener for the dataAvailable event, add an event listener when the Dashboard component gets mounted(), and remove the event listener when the Dashboard component gets destroyed(). Take a look at the code below to see how the event listener is created:

  mounted() {
    // add an event listener to dataAvailable event
    this.establishConnection();
  },
  destroyed() {
    // remove the dataAvailable event listener
    this.destroyConnection();
  },
  methods: {
    destroyConnection() {
      feathersClient.io.off('dataAvailable');
    },
    establishConnection() {
      feathersClient.io.on('dataAvailable', (data) => {
        console.log('Receiving data from server: ', JSON.stringify(data));
        // update variables to the data received from the server
        this.messageCount = data.messageCount;
        this.recentLogins = data.recentLogins;
        this.activeChatRooms = data.activeChatRooms;
        this.openTickets = data.openTickets;
        this.closedTickets = data.closedTickets;
        this.unassignedTickets = data.unassignedTickets;
        this.serverMessage = data;
      });
    },
  },

Now, when you run the vue app and navigate to the /dashboard page, you should see the charts updating every 5 seconds.

Check Your Work

The final code up to this point is on the hello-world branch of this repo: https://github.com/meditatingdragon/realtime-feathers/tree/hello-world.

Level Up: Create a Custom Metrics Service

Go beyond Hello World and create a custom service that delivers the data. Feathers provides an easy way to generate a service for an application feature. For our dashboard, we can create a MetricsService.

feathers generate service

? What kind of service is it? A custom service
? What is the name of the service? metrics
? Which path should the service be registered on? /metrics
? Does the service require authentication? No
   create src/services/metrics/metrics.service.js
    force src/services/index.js
   create src/services/metrics/metrics.class.js
   create src/services/metrics/metrics.hooks.js
   create test/services/metrics.test.js

Define the MetricsService as a custom service that can create data every 5 seconds.

const { getRandomInt } = require("../../utils/dataGenerator");

/* eslint-disable no-unused-vars */
exports.Metrics = class Metrics {
  async create(data) {
    return data;
  }

  setup() {
    let logins = [
      getRandomInt(50, 70),
      getRandomInt(50, 70),
      getRandomInt(50, 70),
      getRandomInt(50, 70),
      getRandomInt(50, 70),
      getRandomInt(50, 70),
      getRandomInt(50, 70),
    ];

    setInterval(() => {
      console.log("Sending new data");
      logins.shift();
      logins.push(getRandomInt(50, 70));
      this.create({
        messageCount: getRandomInt(1000, 10000) + getRandomInt(0, 100),
        activeChatRooms: getRandomInt(5, 100),
        recentLogins: logins,
        openTickets: getRandomInt(0, 100),
        closedTickets: getRandomInt(0, 100),
        unassignedTickets: getRandomInt(0, 100),
      });
    }, 5000);
  }
};

Then we can update our data connection to use the service:

establishConnection() {
    feathersClient.service('metrics').on('created', data => {
        console.log('Receiving data from server: ', JSON.stringify(data));
        // update variables to the data received from the server
        this.messageCount = data.messageCount;
        this.recentLogins = data.recentLogins;
        this.activeChatRooms = data.activeChatRooms;
        this.openTickets = data.openTickets;
        this.closedTickets = data.closedTickets;
        this.unassignedTickets = data.unassignedTickets;
        this.serverMessage = data;
      });
    },
}

Check Your Work

The final code up to this point is on the metrics-service branch of this repo: https://github.com/meditatingdragon/realtime-feathers/tree/metrics-service.

Coming Up Next: Channels

To handle real-time events in future posts, we'll be making use of channels. If you want to get a jump start, take a look at the docs.

Let me know - how will you leverage real-time events in your application?