by Tyler Charbonneau

Kubernetes now runs in more than 70 percent of container environments. Monitoring has become a key way to extract as much information as possible during container runtime. This data is critical when troubleshooting issues. It’s also integral to optimizing performance, both proactively and reactively.

However, Kubernetes presents a unique challenge on two fronts: setup and monitoring. To begin, it’s difficult to really nail your deployment in an organized, high-performing way. Common mistakes involve incorrectly sizing your nodes, consolidating containers, or properly creating namespaces. Making resource requests via a configuration file or kubectl requires strong forethought.

Consider this: roughly 49 percent of containers use under 30 percent of their requested CPU allocation, and 45 percent of containers use less than 30 percent of their allotted memory. Real-time monitoring can help prevent these problems. Idle resources are expensive and don’t provide any real benefit to your ecosystem.

Metrics Tracking

These missteps can require mitigation sooner or later. Accordingly, reliably tracking runtime metrics like latency, CPU utilization, and memory usage is often tricky. Other important metrics include the following:

• Cluster state metrics—like pod health and availability
• Node status—including readiness status, CPU/memory/disk load, and network status
• Pod availability
• Disk utilization

Kubernetes doesn’t always excel at displaying this data in a meaningful and readable way. It’s up to you—the DevOps professional—to piece together bespoke solutions. Designing a custom dashboard is difficult and time-consuming. Thankfully, a number of third-party vendors have created capable visualization tools for Kubernetes. Because these tools are open source, they also interface effectively with some adjacent technologies.

Want to keep your Kubernetes deployment healthy and running? Follow along to learn more about these best open-source Kubernetes dashboards.

Evaluation Criteria

How do you judge what makes a tool favorable? For analysis purposes, we’ll primarily dive into these assessment categories:

• Metrics availability
• Usability
• Setup and maintenance requirements

These comparisons are more nuanced than just comparing hard numbers. It’s important to look at standout features and any features that distinguish one tool from the next. Each solution’s “secret sauce,” as it were, could be uniquely beneficial for your custom deployment. There are also many subjective ways to measure a tool’s worth. Quality of documentation, information presentation, and even graphical user interface (GUI) differences can shape tooling opinions.

This guide focuses predominantly on objective aspects but will introduce other notable characteristics that might sway your decision. Balancing functional needs with personal preferences is critical. Here are some top picks on both the server and client sides:

Server-Side Tools

Many teams might opt for server-side monitoring tools to capture Kubernetes data. Kubernetes natively captures resource utilization data and aggregates it in a database. This is known as the resource metrics pipeline. Both the Horizontal Pod Autoscaler controller and the kubectl-top utility generate this data during usage—which Kubernetes temporarily collects via an in-memory metrics-server. This information is exposed through the metrics.k8s.io API, which allows external services to tap into usage data.

The metrics-server requests all resource metrics from discovered cluster nodes via kubelet. Furthermore, kubelet will dig deeper by translating pods into associated containers—ultimately exposing that information with the Resource Metrics API.

Additionally, DevOps teams can leverage the full metrics pipeline to view more intricate data. This also taps into nodes via kubelet, but either the custom.metrics.k8s.io or external.metrics.k8s.io APIs expose that information instead. Note that Kubernetes can react natively based on these gathered metrics to counteract problems. The onus isn’t entirely on the development side.

Server-based metrics tracking has some significant advantages. This collection method is reliable due to its simplified collection approach; you don’t have to wrestle with thousands of nodes, pods, or containers individually. Because server tracking offloads the burden from your Kubernetes infrastructure, you’ll also see a performance improvement. It’s relatively easy to wrangle your data as required from the server once it’s up and running.

However, no system is perfect, and this fact also applies to server-side monitoring. These solutions can be harder to configure, as you have to install essential components within your system that can effectively transmit data elsewhere. There can be more failure points across the system. Server-side transitions can also be costlier. The overall breadth of the server-side data you collect might be more limited—or at least be missing some crucial insights of interest. Server logs aren’t always human-readable, and logs might only be retained for a limited time unless they’re archived.

That said, millions of users have come to love their server-side monitoring tools. Here are some of the top dogs in the category that have become household names:

Kubernetes Dashboard

Image courtesy of Chuka Ofili.

Offered natively through the web browser, the Kubernetes Dashboard operates as a web app, offering detailed insight into your containerized environment. While some open-source solutions are read-only, the dashboard allows you to deploy, troubleshoot, and actively manage system components. The tool excels at managing both system resources (Jobs, DaemonSets, deployments) and applications. It’s essential to monitor both the system and the microservices that run atop it.

You can also monitor the following via the Dashboard, all accessed easily via the left-hand sidebar:

• Namespaces
• Nodes
• Persistent volumes
• Roles
• Storage classes
• Cron jobs
• Replica sets and replication controllers
• Stateful sets
• Discovery and load balancing parameters
• Configurations and storage setups

Each of these categories has a detailed view with easy-to-read infographics. Because the Kubernetes Dashboard is web-based, you can access it anywhere at any time. This makes it a fantastic, platform-agnostic tool that functions admirably regardless of one’s operating system—or even cluster architecture. To deploy a containerized application, simply connect your YAML or JSON configuration file with the integrated setup wizard.

On the logging side, the built-in Logs Viewer pulls records from containers belonging to single pods and displays them in a list format. That output is organized in chronological order for easier parsing. You can download these specific logs as needed with a single click.

The Dashboard isn’t installed by default. Run the following command to deploy it:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.2.0/aio/deploy/recommended.yaml

You can access Dashboard via kubectl by running the kubectl proxy command. Only machines having executed this command can view the Dashboard UI. Additionally, your deployment will adhere to simple role-based access control (RBAC) standards, exclusively necessitating a Bearer token for successful login.

Skooner (Formerly K8dash)

Having received a face-lift and new name, Skooner continues to be a leading open-source tool for holistically monitoring Kubernetes. The developers behind the project tout the simplicity and real-time availability of their solution—no refreshes or manual polling is required to fetch system data as it’s collected. Additionally, the YAML provided within the tool’s resource repository allows you to start leveraging Skooner in just a minute’s time.

There’s very little of a learning curve involved on the setup side. All you need is a running Kubernetes cluster, a recommended metrics-server installed, and an optional OpenID Connect configuration. Unlike with the Kubernetes Dashboard, you can log into Skooner using one of three methods: a service account token, OpenID Connect (OIDC), or via NodePort. The first is the easiest while the last is the fastest, per the developers. Those favoring OpenID will naturally gravitate toward OIDC. Note that kube proxy cannot be used to access this Dashboard, as the Authorization header is stripped during execution.

Here’s what you can visualize using Skooner:

• Namespaces
• Nodes
• Pods
• Replica sets
• Deployments
• Storage
• RBAC configurations
• Workloads

Skooner relies heavily on metrics-server to pull runtime metrics. Without this component installed, the platform’s functionality will suffer to a degree. It’ll be much more difficult to summon utilization data, for instance, without tapping into that pipeline.

However, Skooner brings its mobile app to the table. It runs on most phones or tablets, allowing you to keep tabs on crucial metrics while you’re on the go. The solution is highly scalable—responding well to changes in Kubernetes system configurations and growth while continuing to grab relevant information reliably.

Prometheus UI

Image courtesy of Christiaan Vermeulen.

Last but not least, Prometheus is a wildly popular open-source tool maintained by the Cloud Native Computing Foundation (CNCF). As such, it enjoys significant backing and development support by the community at large. You’ll notice an immediate GUI difference between this and the other entrants on our list; Prometheus uses a darker palette and arranges its visualizations a little differently. The information is displayed more densely according to endpoint, host, and port. Prometheus helps monitor the following:

• CPU utilization (including core counts)
• RAM usage (including total available)
• SWAP memory usage (from total)
• Root filesystem usage (from total)
• CPU system load (per interval average)
• Uptime

Prometheus stores data as a time series, where metrics have streams of time-stamped values. Each metric also has its own set of labeled dimensions. These are typically stored, though Prometheus can generate temporary series from proprietary PromQL queries. Visually, metrics are commonly displayed as graphs and the like by linking Prometheus with Grafana—which pulls from an assigned data source.

Getting started with Prometheus requires you to install its exporter component on each relevant Kubernetes node. This acts as a service that streams runtime data to the database and dashboard later on. You can set up a multidimensional data model using queries and key-value pairings.

You can install Prometheus using a pre-compiled binary, via Docker images and volumes or configuration management systems like Chef or Ansible. Prometheus is also good at self-monitoring through the use of included APIs. Finally, Prometheus’ massive community can provide help with this task or any other facet of the monitoring process.

Client-Only Tools

As opposed to server-based monitoring, client-only tools are best for teams needing an easy solution without excessive configuration. They’re generally cheaper and have lower barriers to entry. Those running stateless applications in particular might benefit from client-side monitoring, as critical session or resource data isn’t typically stored on the server. Here are our picks:

Lens by Mirantis

As an ops-focused monitoring tool, Lens is a popular Kubernetes integrated development environment (IDE) that acts as a multidisciplinary continuous integration/continuous delivery (CI/CD) platform. The service bundles a contextual terminal with Prometheus-derived statistics while ensuring that logs are easily viewable. Monitored clusters may either be local or external. Accordingly, you can even add a cluster into the mix by importing a kubeconfig file.

Clusters and their tracked metrics are separated into working groups—useful for different teams or for maintaining segregation within complex deployments. You can summon real-time graphs within the Lens dashboard that are tailored to each namespace and resource. Thanks to included RBAC controls, you can define which users can access specific metrics for greater security. Lens remains open source and free to use.

Octant by VMWare

Like the Kubernetes Dashboard, Octant is an open-source web interface for visualizing your clusters and applications. The solution supports multiple plugins via a core gRPC API, making Octant extensible and richly featured. Like other tools, it provides real-time updates on the health and performance of your cluster’s objects plus related objects. This detailed metrics tracking is meant to simplify the debugging process and highlight problems before they become threatening. Building off of kubectl and kustomize, Octant is a simple and reliable tool for managing the Kubernetes system as a whole.

Conclusion

We work in a fast-paced industry, and the tools available for Kubernetes can change quickly. No matter what, your choice of tools will chiefly depend on your needs and the unique deployment you’ve built with Kubernetes. You’ll want to consider your team’s experience and comfort level when choosing an approach—either server-based or client-based. Open-source tools for monitoring Kubernetes are thankfully robust and widespread across the marketplace. If you harness real-time information, keeping your Kubernetes deployment healthy and operative should be much simpler.

Photo by William Warby on Unsplash