A few months back, Qlik announced Qlik Core, a Dockerized version of Qlik’s associative data engine. After playing with Core for a few weeks (the developer version is free…plus, I work for Qlik), I’ve come to realize how awesome the product is for visualizing and analyzing data. So I’m writing this blog to explain, in my humble opinion, who will benefit from Core, and how to start using it.

First of all, why Qlik Core?

Nowadays, there are several visualization libraries, D3 probably being the most widely used library of the bunch. Click here to see a fully baked example of an interactive visualization built on top of D3. For an application like this, static data is packed and loaded together with your application. But there are a few limitations with this approach, especially when the size data is large (for example, greater than 200 MB)…

1. The app takes a lifetime to load.
2. The browser occupies a large amount of memory, which can even cause the browser to crash if the size of data exceeds the limit.
3. It’s not easy to handle filtering and association.
4. It can be difficult to manage state as the data is explored and filtered.
5. It’s difficult to handle dynamic data.

One alternative is to query a database, requesting and receiving data as we go, but this also has limitations. The main drawback is the query latency that the user may experience. Usually, a query based system takes 0.5–3 seconds to respond to user action, which diminishes the user experience and purpose of an interactive dashboard. 😟

Qlik Core to the rescue — but what is Qlik Core?

Irrelevant Paw Patrol picture (I’m a new parent), from here.

Qlik Core gives you direct access to the first data engine of its kind. And “data engine” is neither a data warehouse nor a database. It is a mechanism to handle data association, analytics and state, especially in large volumes of data.

And when you have this data engine at the 🧡 of an application, you don’t need to send a query to receive the data. Even pulling from different sources, Qlik Core can load the data into memory, process and index it, then power web applications reactively. The data becomes an in-memory instance and your web app now connects to Qlik Core via a WebSocket— all calculations and associations are handled remotely.

So, as mentioned, there are many advantages to using Qlik Core, especially if the data volume is large…

1. The application data resides in a server instead of being passed to the frontend. This not only increases usability from the viewer’s perspective, but it also reduces the load of the server, causing both frontend and backend performance to improve significantly.

2. The developer doesn’t need to worry about wiring up filtering and association through their code. All of this, as well as state management, is handled by Qlik Core now and the app’s visualizations and interactivity can simply hook into the engine.

3. On top of all this, Qlik Core is great at loading data from different data sources and meshing them into one.

So, we’re killing several birds with one stone as they say. And now you know why I think this product is amazing (😍 → that’s me building with Qlik Core and I swear it’s not just because I work at Qlik).

Ready to get started?

Hopefully, you're nodding like this.

Qlik Core is pretty new so getting started can be somewhat tricky (we’re working on improving this but in the meantime, please give us feedback). The official documentation for Qlik Core is at https://core.qlik.com/. There you’ll find this tutorial which I find pretty straightforward. Following the instructions will get your Qlik Core instance running quickly, especially if you’re familiar with Docker. There is also a tutorial video created by Qlik Branch (more to come but this our home for all that is developer and Qlik) which goes through the basics.

But what is after this?

Let me show you with some simple examples…

I have built some examples to illustrate how Qlik Core is used. Basically, to use Qlik Core, you need enigma.js (one of Qlik’s open source libraries) to easily communicate with the engine using JavaScript. The only way to talk to Qlik Core is via our Engine API which is a language-agnostic, WebSocket-based API. Enigma.js is simply a wrapper library for this.

So, here are some examples so you can better understand how Qlik Core works…

*️⃣ Hello engine example

*️⃣ Hello data example

*️⃣ Check if a Core instance is healthy

*️⃣ View data model in an app

*️⃣ Upload a pre-built app to Qlik Core

Hopefully, this is enough to get you started learning about and using Qlik Core. Over time, we’ll be adding more tutorials, tips and tricks to help developers not only get started with Qlik Core but also with the APIs themselves.

So keep an eye for that on Knowledge section of Qlik Branch. In the meantime, check out the tutorials there explaining more about our associative data engine, how it works and how to work with it. And if you are really interested in Qlik Core or need help, let me know. Just leave a comment below or join our Qlik Branch Slack (see the #qlik-core channel or DM me [at]wu_zh).

Cheers, happy coding and happy Halloween everyone.

— Wu