Part 10: Visualize data using Kibana Lens

Table of Content

In this blog series, we have created a full stack app (Node.js and React)that enables users to search for earthquake data stored in Elasticsearch.

Searching for data is one way to learn about our dataset.

But we can learn so much more when we can visualize our data.

To wrap up this series, we will learn how to use Kibana Lens to analyze and visualize earthquake data stored in Elasticsearch!

Resources

Would you rather watch a video to learn this content? Click on the link below!

Episode 10: Visualize data with Kibana Lens

What are we creating today?

We will create a Kibana dashboard that contains panels of various data visualizations.

Table

In the upper left corner(red box), we have a table that lists 10 locations with the highest frequency of earthquakes.

Heat Map

To the right(orange box), we have an earthquake heat map. This allows you to view the world map and visualize the location and the frequency of earthquakes by color.

Metrics

In the row below(yellow box), we have high level metrics about our earthquake data.

It shows you the number of earthquakes in our dataset, the maximum magnitude, and the average depth of earthquakes in our data.

Histogram

In the next row, we have the earthquake magnitude histogram(green box). This graph displays different levels magnitude that exist in our dataset and how many earthquakes have exhibited this magnitude.

Donut Chart

To the right, we have a donut chart(blue box) that shows the types and percentage of quakes present in our dataset.

Creating the Dashboard

Let’s go over how we can create this dashboard with Kibana.

Step 1: Open the Kibana home page

From the home page, click on the menu icon(red box). From the drop down menu, click on the Dashboard option(blue box).

Step 2: Create a dashboard

Kibana dashboard lets you easily create views that pull together charts, maps, and filters to display the full picture of your Elasticsearch data(excerpt from Kibana Dashboard page).

Click on the Create a dashboard option(pink box).

It will take you to the dashboard where you can add and edit visualizations.

Step 3: Create a table

The first visualization we will create is a table that lists 10 locations with the highest frequency of earthquakes.

From the dashboard page, click on the Select type option(blue box).

It will display a drop down menu with various visualization tools.

From the drop down menu, choose the Lens option(red box).

Kibana Lens is an intuitive UI that helps you create data visualization with ease!

Whenever possible, start with Lens as you can create majority of the visualizations with this tool.

First thing we must do is to select the correct data source and the time range in which we want to visualize the data. All of the panels in the dashboard will reflect earthquake data that falls within the chosen time range.

Data Source
You will see that the dashboard has already chosen the earthquakes data view(red box) we have created in part 7.

If you have multiple data views, make sure the correct data source is selected.

Time Range
Click on the calendar icon(red box) to display the drop down menu.

In part 7, we have retrieved USGS earthquake data from the past 30 days.

Select the Last 30 days option(green box).

Table
Click on the downward arrow(red box) to display the menu of visualization type. Select the Table option(blue box).

We are creating a table that lists 10 locations with the highest frequency of earthquakes. Since location data is stored in the field place, drag and drop the field place.keyword (red box) into the editor(blue box).

It will automatically create a table for you with two columns(red and blue boxes).

The column Top 5 values of place.keyword(red box) lists 5 locations with the highest document(i.e. earthquake) counts.

The column Count of Records(blue box) lists the number of documents(i.e. earthquakes) that have occurred in that particular location.

Let's look at the table that we want to create.

Our final table contains two columns called Location by frequency and # of earthquakes.

It lists 10 locations with the highest frequency of earthquakes in descending order.

We will change our current table to mirror the final table.

Column: Top 5 values of place.keyword
From the table page, click on the Top 5 values of place.keyword option(blue box).

It will display a pop up menu where you can customize the rows of the table.

Red box
You will see that the rows reflect data from the field place.keyword.

Orange box
The number of values displayed is set to 5.

Change this value to 10 to display 10 locations with the highest frequency of earthquakes.

Yellow and green boxes
The rows are ranked by count of records in descending order.

Blue box
The Name is set to Top 5 values of place.keyword.

Change this value to Location by frequency.

The changes you make will be auto saved. You will also see that the table is auto updated as well.

Next, we will update the column Count of records.

Click on the close option(red box).

Column: Count of records
Click on the Count of records option(red box).

From the pop up menu, change the Name to # of earthquakes(red box).

As you make these changes, you will see that the changes are auto saved and the table is auto updated(blue box).

Click on the Save and Return button(purple box) in the upper right corner.

You will see that the table has been added to the dashboard.

Let's add a title to this table.

Click on the [No Title] section(red box).

In the pop up menu, type in "10 locations with the highest frequency of earthquakes"(red box). Click on the Save button(green box).

You will see that the title(red box) has been added to the table panel.

Step 4: Save and name the dashboard

From the dashboard page, click on the Save button(green box) in the upper right corner.

Name the dashboard to whatever it makes sense to you. I named mine "earthquakes_data_visualization"(red box). Click on the Save button(green box).

You will see that our dashboard is now saved and has been named as earthquakes_data_visualization(red box).

Step 5: Create a heat map

With heat maps, you can view the world map and visualize the location and the frequency of earthquakes by color.

It is an effective way visualize earthquake hotspots around the world and easily detect patterns.

From the dashboard page, click on the select type option(red box) and click on Maps(blue box).

The page will display a world map.

Click on the Add layer button(red box).

Click on the Heat map option(blue box).

Click on the downward arrow(red box) to select the earthquakes data view(blue box) we created in part 7 .

You will see that Lens automatically selected the field coordinates(blue box below). This field contains inner fields lat(latitude) and lon(longtude). These fields were mapped to a geo_point to mark the location of earthquakes.

Click on the Add layer button(red box).

It will display a pop up menu where you can further customize the heat map.

To create the heat map shown in the beginning of this blog, set the resolution to low(blue box) and click on the Save & close option(red box).

You will see that the earthquake locations are now marked with different colors on the map.

Warm colors are associated with the maximum earthquake activity, whereas the cool colors are associated with minimum earthquake activity.

By looking at our heat map, we can see that the states California and Utah seem to have frequent occurrences of quakes.

Click on the Save and return button(red box) in the upper right corner.

You will see that the heat map has been added to our earthquakes_data_visualization dashboard.

Let's give this panel a title.

Click on the [No Title] section(red box).

In the pop up menu, type in "Earthquake heat map"(blue box) and click on the Save button(red box).*

You will see that the panel title(red box) has been added.

Step 6: Create metrics panels

Let's calculate and display high level metrics such as the number of earthquakes, maximum magnitude, and average depth.

From the dashboard, click on the Select type option(red box) and click on the Lens option(blue box).

In the next page, click on the downward arrow(red box) to reveal the Visualization type drop down menu. Scroll down on the menu until you see the Metric option(blue box) then click on it.

Number of earthquakes
The first panel we will create is Number of earthquakes.

The number of documents(i.e. earthquakes) that fall within the chosen time range(last 30 days) will give us this value.

To get this value, drag and drop Records(red box) into the editor(orange box).

You will see that we have a total of 8,181 documents that fall within the last 30 days time range.

Let's rename "Count of records"(red box) to "Number of earthquakes" as shown in the final dashboard.

To make the changes, click on "Count of records"(orange box above) to display the pop up menu.

Under the Name section(green box), type in "Number of earthquakes". Click on the Save and return button(pink box).

You will see that the panel for Number of earthquakes has been added to the dashboard.

Max magnitudes
Let's create a panel that displays the maximum magnitude of earthquake that exist in our data.

From the dashboard, click on the Select type option(red box) and click on the Lens option(blue box).

In the next page, click on the downward arrow(red box) and select the Metric option(blue box).

We are creating a panel for maximum magnitude of earthquakes found in our data set.

Drag the field mag(red box) into the editor(orange box).

You will see that it automatically displays "Median of mag"(red box).

However, we want to display the maximum value of magnitude found in our data set.

Click on "Median of mag"(orange box) in the edit menu to change this setting.

In the pop up menu, click on the function Maximum(green box) to get the Maximum magnitude.

In the Name section(orange box), update the name to "Maximum magnitude".

Click on the Save and return button(pink box) in the upper right corner to add this panel to the dashboard.

You will see that the panel Maximum magnitude has been added to the dashboard and that the maximum magnitude of earthquake in our data set is 7!

Average Depth
Let's create a panel for Average depth of earthquakes in our data set.

From the dashboard, click on the Select type option(red box) and click on the Lens option(blue box).

Click on the downward arrow(red box) and select the Metric option(blue box).

We are creating a panel for average depth of earthquakes in our data.

Drag the field depth(red box) into the editor(orange box).

You will see that it automatically displays the "Median of depth"(red box).

However, we want to display the average depth of earthquakes found in our data set.

Click on "Median of depth"(orange box) in the edit menu to change this setting.

In the pop up menu, click on the function Average(green box) to get the average depth of earthquakes.

In the Name section(orange box), update this to "Average depth (km)".

Click on the Save and return button(pink box) in the upper right corner to add this panel to the dashboard.

You will see that the panel Average depth (km) of earthquakes has been added to the dashboard and that the average depth of earthquake in our data set is 20.46 km!

In our final dashboard, we see that these three metrics panels take up a row.

One of the cool things about the dashboard is that you can change the size of these panels and reorganize the panels in any way you want.

You can adjust the size of the panel by grabbing the lower right corner of the panel(green box) and adjusting the size of the panel.

You can move the panel by clicking the upper part of the panel(blue box) then by holding and dragging the panel wherever you wish.

After resizing and reorganizing your metrics panels, your dashboard should look like the following.

Step 7: Create a magnitude histogram

Next, we will create an earthquake magnitude histogram.

This panel displays different levels magnitude that exist in our dataset and how many earthquakes have exhibited this magnitude.

To create this histogram, click on the Select type option(red box) and select the Lens option(blue box).

Click on the downward arrow(red box) then select the Bar vertical option(blue box).

You will see the following page where you can add a field to a horizontal axis(red box) and vertical axis(blue box).

Let's take a look at the magnitude histogram in the final dashboard.

On the horizontal axis, the levels of magnitude are displayed. On the vertical axis, we have the number of earthquakes that exhibit the corresponding magnitude.

Drag the field mag(green box) into the Horizontal axis section(red box).

Drag Records(orange box) into the Vertical axis section(blue box).

You will see that it automatically creates a histogram for you with Count of records(purple box) on the vertical axis and mag(green box) on the horizontal axis.

Let's rename the vertical axis label Count of records to Number of earthquakes.

Click on Count of records(red box) from the edit menu.

In the Name section(purple box), type in "Number of earthquakes".

Let's also change the color of the histogram to pink as shown in the final dashboard.

From the edit menu. click on the Series color option(orange box).

It will display the color options you can apply to the histogram.

Click on the pink color(black arrow).

You will see that the color of the histogram has been changed to pink.

Let's rename the horizontal axis label mag to Magnitude.

Click on the close option(blue box).

Click on mag(orange box) from the edit menu.

*In the Name section(purple box), type in "Magnitude".

Click on the Save and return button(green box) in the upper right corner.*

You will see that the histogram has been added to the dashboard.

You see many earthquakes exhibiting magnitude level that fall between 0.8-1.2, which are pretty mild earthquakes that are not usually felt.

We also observe fewer number of earthquakes exhibiting higher magnitude levels that cause different levels of damage.

For more information on earthquake magnitude level and its effects, check out this resource!

Let's give this panel a title!

Click on the [No Title]section(orange box).

In the pop up menu, type in "Earthquake magnitude histogram"(green box) then click on the Save button(purple box).

You will see that the panel title has been added.

Step 8: Create a donut chart

Lastly, let's create a donut chart that shows the types and percentage of quakes present in our dataset.

From the dashboard page, click on the Select type option(red box) and click on the Lens option(blue box).

In the next page, click on the downward arrow(red box). Scroll down until you see the Donut option(blue box) and click on it.

Let's look at the final Donut chart.

The donut is divided into types of quakes. Each type is represented by a slice of the donut.

Drag the field type(orange box) and drop it into the Slice by section(blue box).

The size of the slice is determined by how many earthquakes(Records) of each type exists in our data set.

Drag and drop Records(red box) into the Slice by section(green box).

You will see that Lens has automatically created a donut chart that shows the type and percentage of quakes that exists in our data.

As you can see, the majority of the quakes present in our data is of type earthquake(98.22%).

Ice quake makes up 0.01% of our data.

Some of the quakes are man made:

quarry blast(1.22%)
explosion(0.54%)
chemical explosion(0.01%)

Let's create a legend for types of quakes and place it in the upper right side of the donut chart panel.

Click on the legend logo(green box).

It will display a drop down menu where you can edit the legend.

In the Display section, click on the Show option(orange box).

You will see a legend(blue box) show up in the upper right corner.

The Alignment section(purple box) allows you to specify where in the panel the legend should be displayed. Since we want the legend in the upper right corner, leave the default option(right facing arrow).

Click on the Save and return button(red box) in the upper right corner to add this panel to the dashboard.

You will see that the donut chart has been added to the dashboard.

If you hover over a slice(red box), it shows the quake type(quarry blast) this slice represents, how many documents(i.e. quakes) are of this type(100), and what percentage of documents are of this type(1.22%).

Let's give this panel a title.

Click on the [No Title] section(blue box).

In the pop up menu, type in "Quake types"(blue box) and click on the Save button(red box).