Create a Common Lisp Web app using ningle

In this post we are going to build a Common Lisp web application using a light-weight framework called ningle.

For building web applications in Common Lisp using a scalable and conventional framework, we can use Caveman, I have written a post about this here.

Lisp for the Web - 5

Rajasegar Chandran ・ Jul 3 '21

#commonlisp #webdev #lisp #caveman2

We are going to build a small demo app made in Common Lisp to get a list of Beers and their details like food pairings from Punk API which is based on BrewDog's DIY Dog API.

Before diving into the topic, let's first have a brief introduction about the different tools and libraries we are going to use to build the app.

ningle

ningle is a lightweight web application framework for Common Lisp created by Eitaro Fukamachi. It is a fork project of Caveman. It doesn't require you to generate a project skeleton.

As ningle is a thin framework, you need to have subtle knowledge about Clack. It is a server interface ningle bases on.

Clack

Clack is a web application environment for Common Lisp inspired by Python's WSGI and Ruby's Rack. Clack provides a script to start a web server. It's useful when you deploy to production environment. You need to install Roswell before as Clack depends on it.

(defvar *handler*
    (clack:clackup
      (lambda (env)
        (declare (ignore env))
        '(200 (:content-type "text/plain") ("Hello, Clack!")))))

Create project

First, create our project folder by creating a new directory and add the relevant source files. In our case we only need one file called app.lisp.

mkdir cl-beers
cd cl-beers
touch app.lisp

Installing dependencies

Second, let's install and load the required dependencies for our project using quicklisp. Let's add the below line at the top of our app.lisp

(ql:quickload '(:ningle :djula :dexador :cl-json))

A normal web application framework need to take care of some minimal things to get it running.

Those are:

Template for rendering HTML markup with dynamic content
AJAX requests
Handle JSON
Able to define routing with parameters and various HTTP methods

Let's see how we can implement each of one these in our demo application using some third-party libraries and dependencies with ningle.

Templating with Djula

Djula is a port of Python's Django template engine to Common Lisp. Caveman framework by default uses this template engine for rendering the HTML.

Create a new folder called templates inside our project folder. And we need to setup our template folder with some default layout and the individual template files for our pages or routes.

mkdir templates
cd templates
touch index.html
mkdir layouts
cd layouts
touch default.html

Our default template is the master layout for our app and will have markup something like below:

<!doctype html>
<html lang="en">
  <head>
    <meta charset="UTF-8"/>
    <title>cl-beers</title>
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.0.2/dist/css/bootstrap.min.css" rel="stylesheet" integrity="sha384-EVSTQN3/azprG1Anm3QDgpJLIm9Nao0Yz1ztcQTwFspd3yD65VohhpuuCOmLASjC" crossorigin="anonymous">
  </head>
    <body>
      {% block content %}
      {% endblock %}
    <script src="https://unpkg.com/htmx.org@1.5.0"></script>
    <script src="https://unpkg.com/hyperscript.org@0.8.0"></script>
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.0.2/dist/js/bootstrap.min.js" integrity="sha384-cVKIPhGWiC2Al4u+LWgxfKTRIcfu0JTxR+EQDz/bgldoEyl4H0zUF0QKbrJ0EcQF" crossorigin="anonymous"></script>
  </body>
</html>

Now we need to wire up the template engine Djula to our app by telling the system where to find the template files, how to load them and render them.

;; Tell Djula to load the templates from the templates directory
(djula:add-template-directory  #P"templates/")
;; Set up the template registry for Djula
(defparameter *template-registry* (make-hash-table :test 'equal))

;; render template - copied & modified from caveman
(defun render (template-path &optional data)
  (let ((template (gethash template-path *template-registry*)))
    (unless template
      (setf template (djula:compile-template* (princ-to-string template-path)))
      (setf (gethash template-path *template-registry*) template))
    (apply #'djula:render-template* template nil data)))

AJAX requests with dexador

Dexador is a HTTP client for Common Lisp with neat APIs and connection-pooling. It is fast, particularly when requesting to the same host with neat APIs and signal a condition when HTTP request failed.

This is how you use dexador to make GET requests:

(dex:get "https://api.punkapi.com/v2/beers?per_page=24")

JSON parsing with cl-json

We need our app to handle JSON responses from the APIs, and with Lisp there is an awesome library called cl-json for the job. cl-json provides an encoder of Lisp objects to JSON format and a corresponding decoder of JSON data to Lisp objects.

We can use the decode-json-from-string function from cl-json to decode the response from the API.

(let ((beers (cl-json:decode-json-from-string (dex:get "https://api.punkapi.com/v2/beers?per_page=24"))))

Defining routes

Now we move on to defining routes for our app. In ningle we will just update the ningle:route object with the url and a lambda function which takes the parameters for the route as an argument and will handle the route by making some api calls and finally rendering the corresponding template for the route. This is how we define routes in a ningle app.

;; GET /
(setf (ningle:route *app* "/")
      #'(lambda (params)
      (let ((beers (cl-json:decode-json-from-string (dex:get "https://api.punkapi.com/v2/beers?per_page=24"))))
      (render #P"index.html" (list :beers beers)))))

For rendering the templates, we will make use of the render function from Djula which will take the name of the template file and a list object for the template data as arguments.

Running the app

You need roswell to install clack

ros install clack

And then using clackup

clackup app.lisp

Go to http://localhost:5000 in your browser to see the app in action.

Source code

This is the full code for our app.

(ql:quickload '(:ningle :djula :dexador :cl-json))
(djula:add-template-directory  #P"templates/")
(defparameter *template-registry* (make-hash-table :test 'equal))

;; render template - copied & modified from caveman
(defun render (template-path &optional data)
  (let ((template (gethash template-path *template-registry*)))
    (unless template
      (setf template (djula:compile-template* (princ-to-string template-path)))
      (setf (gethash template-path *template-registry*) template))
    (apply #'djula:render-template* template nil data)))

(defvar *app* (make-instance 'ningle:app))

(djula:def-filter :truncate-desc (val)
  (if (> (length val) 100)
      (concatenate 'string (subseq val 0 97) "...")
      val))

(defun increment-page (page)
  (1+ (parse-integer page)))

;; GET /
(setf (ningle:route *app* "/")
      #'(lambda (params)
      (let ((beers (cl-json:decode-json-from-string (dex:get "https://api.punkapi.com/v2/beers?per_page=24"))))

      (render #P"index.html" (list :beers beers)))))

;; GET /more
(setf (ningle:route *app* "/more")
      #'(lambda (params)
      (print params)
      (let* ((page (cdr (assoc "page" params :test #'string=)))
         (beers (cl-json:decode-json-from-string (dex:get (concatenate 'string "https://api.punkapi.com/v2/beers?per_page=24&page=" page)))))

      (render #P"_more-beer.html" (list :beers beers :page (increment-page page))))))

;; GET /beer/:id
(setf (ningle:route *app* "/beer/:id")
      #'(lambda (params)
      (let ((beer (cl-json:decode-json-from-string (dex:get (concatenate 'string "https://api.punkapi.com/v2/beers/" (cdr (assoc :id params)))))))

        (render #P"show.html" (list :beer (car beer))))))

;; GET /random
(setf (ningle:route *app* "/random")
      #'(lambda (params)
      (let ((beer (cl-json:decode-json-from-string (dex:get "https://api.punkapi.com/v2/beers/random" ))))
        (render #P"show.html" (list :beer (car beer))))))

;; GET /glossary
(setf (ningle:route *app* "/glossary")
      (render #P"glossary.html"))

;; POST /search
(setf (ningle:route *app* "/search" :method :POST)
      #'(lambda (params)
          (let* ((query (cdr (assoc "query" params :test #'string=)))
                (beers (cl-json:decode-json-from-string (dex:get (concatenate 'string "https://api.punkapi.com/v2/beers?beer_name=" query)))))

            (render #P"_search-results.html" (list :beers beers)))))

*app*

The source code for this tutorial is hosted here in Github

Hope you have enjoyed the post, please let me know your comments and feedback in the comments section below. I have omitted a lot of implementation details here, since explaining each and every line is not possible within a single blog post. You can take a look at the source code and figure it out yourself.

Blog