Lua Script in Redis using node_redis

Redis allows us to upload and execute Lua scripts on the server. As these scripts are executed on the server, it makes the reading and writing data very efficient. Lua scripts behave like stored procedures for Redis. where it let us run our application logic inside Redis.

Redis allows Lua scripts to have Blocking semantics which means when it is invoked blocks the execution of its invoker, Hence making the execution of Lua scripts atomic in nature.

Lua scripts are considered a part of the client-side application. It executes on the server but they are not named, versioned, or persisted. As a result, these scripts can be missing from the Redis server, and it is up to the application code to reload these scripts.

Lua Programming Language

Lua is a powerful and fast programming language that is easy to use and embed into your application. It's lightweight and portable with only 21 keywords.

Lua provides a single data structure which is tables. All data structures like arrays, records, lists, queues, and sets are represented with tables in Lua.

backend_frameworks = {"django","express","flask"}

Installing Lua Programming

For installing Lua in ubuntu
For installing Lua in MacOS

Example of Lua programming language

function sum(a, b)
    return a + b
end

local a = 1
local b = 2
local c = sum(a, b)
print(c)

Lua script and Redis

Redis provides EVAL Command, which allows execution on Lua scripts.

Examples

EVAL "return redis.call('get', KEYS[1])" 1 key

In this example, We can see the EVAL command is used to execute Lua script. It takes several arguments a script, numkey, keys, and name. numkey is the number of keys and names to be taken. These keys and names are additional inputs and are not necessarily required. these keys and names provided in arguments are available to the script through KEYS and ARGV global runtime variables.

EVAL "return { KEYS[1], KEYS[2], ARGV[1], ARGV[2], ARGV[3] }" 2 key1 key2 arg1 arg2 arg3

Interacting with Redis Commands from a Lua script

Lua script uses redis.call() or redis.pcall() function to interact with redis.

Example of SET command

EVAL "return redis.call('SET', KEYS[1], ARGV[1])" 1 key1 val1

redis.call() function take SET command, and executes it with given key and name arguments.

Loading and executing Lua script

Redis provides a SCRIPT LOAD command for caching Lua script. It returns a sha1 hash , which can be used to execute the stored scripts. SCRIPT LOAD command doesn't validate the script. For executing cached script Redis provides the EVALSHA command.

SCRIPT LOAD "return { KEYS[1] , ARGV[1] }"


SCRIPT LOAD "returns { KEYS[1] , }"

Now, both scripts are cached inside Redis and return a sha1 hash.

EVALSHA 9e9ef47970b6b31a079e3f4d4bd2e755c53ba3e0 1 key1 val1

scripts stored in Redis are not persisted but removed explicitly. We can use the SCRIPT FLUSH command in Redis to allow the flushing of all stored scripts. it has two modes which are Async and Sync.

ASYNC: flushes the cache asynchronously
SYNC: flushes the cache synchronously

SCRIPT FLUSH ASYNC

Running Lua Script File Using redis-cli

Most of the time our Lua script will be in the .lua file and we will need to store and execute that file directly using redis-cli.

Syntax:

redis-cli --eval fileName.lua keys , argv

Example:

This Lua script is for setting key-value pairs in a set if it doesn't exist. For this, we use the GET command for getting the value if we didn't get the value we will use the SET command to put this key value in the Redis set.

local key = KEYS[1];
local value = ARGV[1];

local oldValue = redis.call('GET', key);

if (oldValue == false) then
  redis.call('SET', key, value);
  return {key, value};
else
  return {key, oldValue};
end

Executing this script

redis-cli --eval ./addKeyValue.lua setKey1 , value2

Debugging this script

Redis also provides a Lua debugger. To use it, pass the --ldb flag.

redis-cli --ldb --eval ./addKeyValue.lua setKey1 , value2

Caching this script

We can simply use the SCRIPT LOAD command to store scripts in Redis

redis-cli SCRIPT LOAD "$(cat ./addKeyValue.lua)"

Executing this Stored script

redis-cli EVALSHA "5828794b7233cb735e77a5346aab6384befa6179" 1 "key1" "val1"

Managing Lua Scripts for Redis in Node

There can be many ways to use Lua scripts with our nodeJS application.

Few of which are :

Using node_redis " scriptLoad" and " evalSha" Commands
Using " eval" Command

Before exploring these methods for executing Lua scripts from our nodeJS application. First, let's create the folder structure. I mostly keep all my Lua scripts inside a folder called luaScripts inside my project directory.

Lua scripts - addKeyIfNotExists.lua , we are using

local key = tostring(KEYS[1]);
local value = tostring(ARGV[1]);

local oldValue = redis.call('GET', key);

if (oldValue == false) then
  redis.call('SET', key, value);
  return {key, value};
else
  return {key, oldValue};
end

Let's create an HTTP webserver with a Redis connection

var http = require('http');
var fs = require('fs');
var redis = require('redis');
var client = redis.createClient();

client.connect();
client.on('ready', () => console.log('Redis is ready'));
client.on('error', (err) => console.log('Error ' + err));

var PORT = process.env.PORT || 3000;

var server = http.createServer(function (req, res) {
  res.writeHead(200, { 'Content-Type': 'text/html' });
  client.GET('key').then((reply) => {
    res.write(` hello ${reply}`);
    res.end();
  }).catch((err) => {
    console.log(err);
    res.write(err);
    res.end();
  });
});

server.listen(PORT, function () {
  console.log(`Server listening on port ${PORT}`);
});

Using node_redis " scriptLoad" and " evalSha" Commands

Creating a Lua variable of type dictionary to store Script and Script Hash.

var lua = {
  script: fs.readFileSync('./luaScripts/addKeyIfNotExists.lua', 'utf8'),
  sha: null
};

Modifying our nodeJS script to catch Lua script into Redis DB

server.listen(PORT, function () {
  client.scriptLoad(lua.script).then(sha => {
    lua.sha = sha;
    console.log(`Script loaded ${sha}`);
    console.log(`Server listening on port ${PORT}`);
  }).catch(err => {
    console.log(err);
  });
});

while starting the web server, we will use the scriptLoad command of node_redis, to load our Lua script and store the hash in lua.sha variable.

For using this cached script, we will use the evalSha Redis command.

client.evalSha(lua.sha, {
      keys: ['key1'],
      args: ['value1']
    }).then((reply) => {
      res.write(` key : ${reply[0]} value: ${reply[1]}`);
      res.end();
    }).catch((err) => {
      console.log(err);
      res.write(err);
      res.end();
    });

Complete nodeJS Scripts

var http = require('http');
var fs = require('fs');
var redis = require('redis');
var client = redis.createClient();

client.connect();
client.on('ready', () => console.log('Redis is ready'));
client.on('error', (err) => console.log('Error ' + err));

var PORT = process.env.PORT || 3000;

var lua = {
  script: fs.readFileSync('./luaScripts/addKeyIfNotExists.lua', 'utf8'),
  sha: null
};

var server = http.createServer(function (req, res) {
  res.writeHead(200, { 'Content-Type': 'text/html' });
  if (req.url === '/add') {
    client.evalSha(lua.sha, {
      keys: ['key1'],
      args: ['value1']
    }).then((reply) => {
      res.write(` key : ${reply[0]} value: ${reply[1]}`);
      res.end();
    }).catch((err) => {
      console.log(err);
      res.write(err);
      res.end();
    });
  } else {
    res.write('<h1>Hello World</h1>');
    res.end();
  }
});

server.listen(PORT, function () {
  client.scriptLoad(lua.script).then(sha => {
    lua.sha = sha;
    console.log(`Script loaded ${sha}`);
    console.log(`Server listening on port ${PORT}`);
  }).catch(err => {
    console.log(err);
  });
});

Using "eval" Command

For testing and debugging, we can use eval Command. This command takes three parameters Lua Script, keys and args.

  client.eval(lua.script, {
      keys: ['key1'],
      args: ['value1']
    }).then((reply) => {
      res.write(` key : ${reply[0]} value: ${reply[1]}`);
      res.end();
    }).catch((err) => {
      console.log(err);
      res.write(err);
      res.end();
    });