TheLazy.Dev
Portfolio GitHub

Blog

clojure |library |clojars |maven

Creating and publishing Clojure libraries

kozieiev

Volodymyr Kozieiev

Posted on October 31, 2021

Creating and publishing Clojure libraries

Table of contents:

Here we are going to create a Clojure library, pack it to a JAR file, deploy it to a Maven repository and make its code available from Clojure and ClojureScript projects. We will create the simplest project and will grow it step by step. At the end, we will discuss the build-clj tool that can be used to create the library template projects.

Video version of this article

Notes for readers

Projects here are organized using deps.edn configuration files.

All examples in this post use single-segment namespaces, like (ns basic-math). It makes examples simpler but please don't use this approach for production code. Both Clojure and ClojureScript have subtle issues with single-segment namespaces. Also this can lead to name clashes when your library published and imported along with another library with the same name. So please use something like (ns basic-math.core) or (ns your-name/basic-math).

At the time of writing Clojure CLI with version 1.10.3.986 was used on Mac.

Forms and locations of Clojure libraries

Clojure library can exist in a form of a source code or as a JAR file. JAR stands for "java-archive" and it is a zip-archive with all the library files - sources and resources.

tools.deps library that is used behind the scenes when we invoke clj command has a concept of "procurer". Each procurer knows how to download the library and its dependencies. Currently, three procurers exist: local, git and mvn.

local procurer allows you to use a library from a local disk. It can be a folder with a source code or a JAR file.

git procurer allows you to download a library source code from a git repository. Downloaded libraries will be stored in a local folder ~/.gitlibs by default.

mvn procurer gives access to the Maven repositories. These are storages of files widely used in a java-world. Libraries are stored there as JARs. When the library is downloaded from a remote Maven repo it is cached in a local repo on your disk. By default, it lives in ~/.m2/repository. Library JAR should be published to a Maven repo along with the pom.xml file. It is a configuration file for the Maven that contains information about an artifact, for example, its name and version. More information about the POM files can be found in the official documentation.
Later in this article we will use Maven repository called Clojars. It is a community repo for open-source Clojure libraries.

Creating local library

The simplest form of a library is a local folder with a source code. It can be referenced from another project by a relative path.

As an example let's create a test-lib that defines the simple sum function.

The folder structure looks like this:

playground
└── test-lib
    ├── deps.edn
    └── src
        └── basic_math.clj
Enter fullscreen mode Exit fullscreen mode

test-lib/basic_math.clj:

(ns basic-math)

(defn sum [a b]
  (+ a b))
Enter fullscreen mode Exit fullscreen mode

test-lib/deps.edn contains an empty map:

{}
Enter fullscreen mode Exit fullscreen mode

Another project in a playground folder will be the one that uses the test-lib. Let's call it adder. Its main function will take arbitrary number of arguments, sum them and print the result.

Updated folder structure now looks like this:

playground
├── adder
│   ├── deps.edn
│   └── src
│       └── core.clj
└── test-lib
    ├── deps.edn
    └── src
        └── basic_math.clj
Enter fullscreen mode Exit fullscreen mode

adder/core.clj references basic-math namespace from test library, uses the sum function and prints the sum of arguments:

(ns core
  (:require basic-math))

(defn -main [& args]
  (println "Args sum: " (reduce basic-math/sum (map #(Integer/parseInt %) args))))
Enter fullscreen mode Exit fullscreen mode

Here is the content of deps.edn:

{:paths ["src"]
 :deps {test-lib/test-lib  {:local/root "../test-lib"}}}
Enter fullscreen mode Exit fullscreen mode

To make basic-math namespace available, test-lib added under :deps key and pointed to a local folder ../test-lib.

Note that lib name should be qualified, so if you write {test-lib {:local/root "../test-lib"}}, you'll get a warning. At the same time the name itself is not really important because later in the code we won't use it, we use only name of the namespace. So you can write {any-name/any-name {:local/root "../test-lib"}} and that won't be an error.

To check that library successfully used we can switch to adder folder and run main function of core namespace with couple arguments.

$ cd adder
$ clj -M -m core 1 2 3
Args sum:  6
Enter fullscreen mode Exit fullscreen mode

Creating git-based library

Libraries can be stored in a git. Let's push the lib from the previous section to the github and reuse it from there.

First, let's create an empty github repo named clojure-test-lib. After that we need to push there an already existing code of our library.

Here is a series of commands to do this. Note: you have to be inside the test-lib folder when invoking them.

$ cd test-lib
$ git init
$ git add deps.edn
$ git add src/*
$ git commit -m "init"
$ git branch -M main
$ git remote add origin https://github.com/YOUR-GITHUB-NAME/clojure-test-lib.git
$ git push -u origin main
Enter fullscreen mode Exit fullscreen mode

To use the library from the github we need to update an adder/deps.edn file. :git/sha value can be taken from github. It specifies revision of library you want to use:

{:paths ["src"]
 :deps {io.github.YOUR-GITHUB-NAME/clojure-test-lib {:git/sha "4c42a56d9dec002d5a198b61a5d8dcc30b69d3dc"}}}
Enter fullscreen mode Exit fullscreen mode

By contrast with referencing local libraries, here the library name is important. From that name the git url will be deducted for downloading the source code. List of possible library name formats can be found in the official documentation.

Now when we run the updated adder app, we see the notification that our library was downloaded from git:

$ cd adder
$ clj -M -m core 1 2 3
Cloning: https://github.com/YOUR-GITHUB-NAME/clojure-test-lib.git
Checking out: https://github.com/YOUR-GITHUB-NAME/clojure-test-lib.git at 4c42a56d9dec002d5a198b61a5d8dcc30b69d3dc
Args sum:  6
Enter fullscreen mode Exit fullscreen mode

We can use tags to make referencing of particular library version easier. Let's create one by running commands:

$ cd test-lib
$ git tag -a 'v0.0.1' -m 'initial release' 
$ git push --tags
Enter fullscreen mode Exit fullscreen mode

Now adder/deps.edn can be extended with :git/tag key referencing our newly created tag:

{:paths ["src"]
 :deps {io.github.YOUR-GITHUB-NAME/clojure-test-lib {:git/tag "v0.0.1" :git/sha "4c42a5"}}}
Enter fullscreen mode Exit fullscreen mode

Note that :git/sha key still should exist but it can contain a brief version of sha just to make sure that tag wasn't moved to other commit.

Packaging library to a JAR file

Not so long ago you had to use 3rd-party tools to build JAR for own Clojure library. But now official tools.build can be used to create project artificats.

tools.build itself is a Clojure library and it provides API that allows developers to run build actions from Clojure code. It is the core idea of the project that building process should be also written in Clojure and build scripts are just Clojure scripts. Examples can be found in an official documentation.

Let's use tools.build to create JAR for our test-lib project.

First, we need to add build library to test-lib/deps.edn. That file had empty map before and now should look like this:

{:aliases
 {:build {:deps {io.github.clojure/tools.build {:git/tag "v0.6.2" :git/sha "226fb52"}}
          :ns-default build}}}
Enter fullscreen mode Exit fullscreen mode

Here we created build alias that uses dependency on tools.build when invoked. That alias should be used as a tool, with -T option when running clj.

By providing :ns-default build we tell that all commands passed to this alias will be found in the build namespace. So if we run clj -T:build clean, the (clean) function from build.clj file will run.

Now it is time to create the build.clj file itself. It is placed in a root of test-lib project:

playground
├── adder
│   └── ...
└── test-lib
    ├── build.clj      <---
    ├── deps.edn
    └── src
        └── basic_math.clj
Enter fullscreen mode Exit fullscreen mode

Here is the content of build.clj:

(ns build
  (:require [clojure.tools.build.api :as b]))

(def lib 'com.github.YOUR-GITHUB-NAME/clojure-test-lib)
(def version (format "0.0.%s" (b/git-count-revs nil)))
(def class-dir "target/classes")
(def basis (b/create-basis {:project "deps.edn"}))
(def jar-file (format "target/%s-%s.jar" (name lib) version))

(defn clean [_]
  (b/delete {:path "target"}))

(defn jar [_]
  (b/write-pom {:class-dir class-dir
                :lib lib
                :version version
                :basis basis
                :src-dirs ["src"]})
  (b/copy-dir {:src-dirs ["src" "resources"]
               :target-dir class-dir})
  (b/jar {:class-dir class-dir
          :jar-file jar-file}))
Enter fullscreen mode Exit fullscreen mode

At the top of the file we require an API from tools.build library. Than go few declared vars:

  • lib - fully qualified name of our library. Note: here we used com.github.YOUR-GITHUB-NAME as an organization part of the name. But you are free to use any, without mentioning github.
  • version - lib's version. In this code versioning depends on amount of git commits made so far. So after every commit your version will bump.
  • class-dir - folder were we put all files that will go to a JAR file
  • basis - it is a clojure map with all project settings obtained from project deps.edn and its parents. Probably the most important among them are classpath and dependencies.
  • jar-file - name of result JAR file

There are two declared functions in a build.clj. (clean) removes our build directory. And (jar) does the main work - fills the build directory with all required files and creates a JAR file with our library.

There are 3 calls to the tools.build inside the (jar):

  • (b/write-pom) - creates a pom.xml file inside the build directory.
  • (b/copy-dir) - copies the source files and the resources (if we'd have any) to a build directory
  • (b/jar) - packs the content of a build directory to a JAR file

Now when everything is ready we can generate JAR file for our library with commands:

$ cd test-lib
$ clj -T:build jar
$ ls target
classes                    clojure-test-lib-0.0.1.jar
Enter fullscreen mode Exit fullscreen mode

Our library now stored in test-lib/target/clojure-test-lib-0.0.1.jar

Lets change adder/deps.edn to use the library from the local jar:

{:paths ["src"]
 :deps {com.github.YOUR-GITHUB-NAME/clojure-test-lib {:local/root "../test-lib/target/clojure-test-lib-0.0.1.jar"}}}
Enter fullscreen mode Exit fullscreen mode

And run the adder again to make sure everything works:

$ cd adder
$ clj -M -m core 1 2 3
Args sum:  6
Enter fullscreen mode Exit fullscreen mode

Image description

Installing JAR to local Maven repo

Now we have a JAR file but referencing it by relative path doesn't look nice. Ideally we'd like to have it in some remote Maven repo. But before lets install it in a local repo - ~/.m2/repository.

To do this we'll add a new function to the test-lib/build.clj called (install):

; ... previous content of build.clj ...

(defn install [_]
  (b/install {:basis      basis
              :lib        lib
              :version    version
              :jar-file   jar-file
              :class-dir  class-dir}))
Enter fullscreen mode Exit fullscreen mode

Inside our new function we call a function with the same name from the tools.build library that does actual work.

Now we can create and install JAR of our test library with commands:

$ cd test-lib
$ clj -T:build clean
$ clj -T:build jar
$ clj -T:build install
Enter fullscreen mode Exit fullscreen mode

To check that the library was added to the local repo we can inspect the local folder ~/.m2/repository/com/github/YOUR-GITHUB-NAME/clojure-test-lib/. There should be our JAR inside the folder for version 0.0.1.

Now adder/deps.edn should be changed to use library from maven repo instead of local JAR:

{:paths ["src"]
 :deps {com.github.YOUR-GITHUB-NAME/clojure-test-lib {:mvn/version "0.0.1"}}}
Enter fullscreen mode Exit fullscreen mode

Let's run adder again to confirm that changes work:

$ cd adder
$ clj -M -m core 1 2 3
Args sum:  6
Enter fullscreen mode Exit fullscreen mode

Installing JAR to remote repo

Now it is a time to make our extremely useful library publicly available. We want to push it to the Clojars. As you probably already guessed, we are going to add a new function to deps-edn/build.clj that will be responsible for this action. It will be named (deploy).

Unfortunately the standard tools.build library that we were using before doesn't provide functionality to deploy to Clojars. So we will use a 3rd-party library called deps-deploy.

Let's add it as another dependency of the build alias in the test-lib/deps.edn.

{:aliases
 {:build {:deps {io.github.clojure/tools.build {:git/tag "v0.6.2" :git/sha "226fb52"}
                 slipset/deps-deploy {:mvn/version "RELEASE"}}                            ; <-- new dependency
          :ns-default build}}}
Enter fullscreen mode Exit fullscreen mode

Changes to the test-lib/build.clj add the new (deploy) function:

(ns build
  (:require [clojure.tools.build.api :as b]
            [deps-deploy.deps-deploy :as dd])) ; <--- don't foget to add a new require

; ... previous content of build.clj ...

(defn deploy [_]
  (dd/deploy {:installer :remote
              :artifact jar-file
              :pom-file (b/pom-path {:lib lib :class-dir class-dir})}))
Enter fullscreen mode Exit fullscreen mode

As you can see, our (deploy) function relies on the function with the same name from deps-deploy package. Being invoked it will take the JAR file and pom.xml and publish them to Clojars. (b/pom-path) is a helper function from tools.build that retuns path to the pom.xml.

Before you will be able to publish anything to the Clojars, you need to go to clojars.org, create an account there and add a "deploy token". It is used in a place of a password when deploying from the command line.

When you've created an account and a deploy token, deploying the lib will be as simple as running our newly created build command with couple additionl environment variables set:

$ env CLOJARS_USERNAME=USER-NAME CLOJARS_PASSWORD=CLOJARS_XXXXXXXX clj -T:build deploy
Enter fullscreen mode Exit fullscreen mode

CLOJARS_USERNAME - your Clojars account name

CLOJARS_PASSWORD - your created deploy token. Not a login password.

After executing previous command you should see the shell messages saying that *.jar and *.pom were successfully uploaded to Clojars. Also you will see a new library on the Clojars dashboard.

adder/deps.edn doesn't need to be changed, because the way we reference test-lib dependency is the same for local and remote Maven repos.

Modifying library to be suitable for ClojureScript

Our test-lib contains pure Clojure code and doesn't depend on anything Java-specific, so it would be nice to make it available for ClojureScript as well.

At first, let's create a simple ClojureScript project called adder-cljs and try to use test-lib as is.

Here is a structure of the new project created in the playground folder on the same level as the already existing adder and test-lib projects:

adder-cljs
├── deps.edn
└── src
    └── core.cljs
Enter fullscreen mode Exit fullscreen mode

core.cljs implements the same logic that we had before in the adder/src/core.clj. The only difference is that we use the js/parseInt for turning arguments to integers:

(ns core
  (:require basic-math))

(defn -main [& args]
  (println "Args sum: " (reduce basic-math/sum (map js/parseInt args))))
Enter fullscreen mode Exit fullscreen mode

adder-cljs/deps.edn contains dependencies for ClojureScript itself and our test-lib:

{:deps {org.clojure/clojurescript        {:mvn/version "1.10.879"}
        com.github.YOUR-GITHUB-NAME/clojure-test-lib {:mvn/version "0.0.1"}}}
Enter fullscreen mode Exit fullscreen mode

Command for running the script from a terminal will be:

$ clj -M -m cljs.main -re node -m core 1 2 3
Enter fullscreen mode Exit fullscreen mode

And as a result we will get an exception:

Unexpected error (ExceptionInfo) compiling at (REPL:1).
No such namespace: basic-math, could not locate basic_math.cljs, basic_math.cljc, or JavaScript source providing "basic-math" (Please check that namespaces with dashes use underscores in the ClojureScript file name) in file core.cljs
Enter fullscreen mode Exit fullscreen mode

It says that basic-math can't be found because it is not in a .cljs or *.cljc file. As we remember test_lib keeps this namespace in the basic_math.clj. **Fortunately the only change we need to make clojure code usable from both Clojure and ClojureScript is simply store it in *.cljc file.* More sophisticated libraries can also use "reader conditionals" to add the Clojure- or ClojureScript-specific expressions to the code. More info can be found in the official documentation.

So let's rename test-lib/src/basic_math.clj to test-lib/src/basic_math.cljc and publish a new version of the library:

$ cd test-lib
$ mv src/basic_math.clj src/basic_math.cljc
$ git add .
$ git commit -m "make library usable for ClojureScript"
$ clj -T:build clean
$ clj -T:build jar
$ env CLOJARS_USERNAME=USER-NAME CLOJARS_PASSWORD=CLOJARS_XXXXXXXX clj -T:build deploy
Enter fullscreen mode Exit fullscreen mode

As you remember, build.clj file of our library uses commits count for changing version name. This is why we've made a commit here before deploying the library.

As a result, version 0.0.2 of our library now should be pushed to the Clojars.

To use it let's change adder-cljs/deps.edn and bump library version there:

{:deps {org.clojure/clojurescript        {:mvn/version "1.10.879"}
        com.github.YOUR-GITHUB-NAME/clojure-test-lib {:mvn/version "0.0.2"}}}   ; !!!
Enter fullscreen mode Exit fullscreen mode

And run adder-cljs again:

$ clj -M -m cljs.main -re node -m core 1 2 3
Args sum:  6
Enter fullscreen mode Exit fullscreen mode

Adding pom.xml to library root

Up to now the pom.xml file for our library was generated automatically by a call to the (write-pom) function in the test-lib/build.clj and stored in the temporary target folder. Drawback of this is that we can't add any persistent changes to this file because they will be wiped out when clj -T:bulid clean invoked. But fortunately (write-pom) is clever enough and if you have some pom.xml file in the root of the project, it will be taken as a basis and merged with a generated one.

Let's use this feature to add a description and a homepage url to our library so they can be seen on the Clojars page.

First, let's go to the test-lib directory and copy an already generated pom.xml to the root of a project for future modification:

$ cd test-lib
$ cp target/classes/META-INF/maven/com.github.YOUR-GITHUB-NAME/clojure-test-lib/pom.xml .
Enter fullscreen mode Exit fullscreen mode

Now we are going to modify the new pom.xml. We want to replace the actual version of the library with "VERSION" placeholder just to avoid future confusion. Also the description and url fields need to be added. Here is an updated content of the pom.xml, slightly truncated for readability:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <packaging>jar</packaging>
  <groupId>com.github.YOUR-GITHUB-NAME</groupId>
  <artifactId>clojure-test-lib</artifactId>
  <version>VERSION</version>                                               <!-- changed -->
  <name>clojure-test-lib</name>
  <description>Test library for article on Clojure libraries</description> <!-- added -->
  <url>https://github.com/YOUR-GITHUB-NAME/clojure-test-lib</url>          <!-- added -->
  .....
</project>
Enter fullscreen mode Exit fullscreen mode

And now let's commit these changes and push the updated library to the Clojars:

$ cd test-lib
$ git add .
$ git commit -m "Added root pom.xml with description and homepage url"
$ clj -T:build clean
$ clj -T:build jar
$ env CLOJARS_USERNAME=USER-NAME CLOJARS_PASSWORD=CLOJARS_XXXXXXXX clj -T:build deploy
Enter fullscreen mode Exit fullscreen mode

Now, when we go to Clojars, on the page of our library we can see that the version number was bumped, description added and there is a link to our homepage. That means that the url and description fields of root pom.xml were successfully used and merged with the generated fields like version.

Creating library project with clj-new

So far to setup the test-lib project we've made the following:

  • added dependencies for tools.build and deps-deploy
  • created the build.clj with own build commands
  • added the root pom.xml file

These steps aren't unique for the test-lib and you'll need to repeat them again when creating another libraries. Fortunately, there is a project called clj-new that will save you from doing similar work again and again.

With the clj-new you can generate a project template that already equipped with all the features we've discussed and even more.

Here we'll cover the basic steps to create a library with clj-new but please go and read the official docs for the detailed instructions and up to date version of the library.

First, we need to install the clj-new as a tool to make it available for use:

$ clojure -Ttools install com.github.seancorfield/clj-new '{:git/tag "v1.2.362"}' :as clj-new
Enter fullscreen mode Exit fullscreen mode

Now, to create a library we need to run the command:

$ clojure -Tclj-new lib :name com.github.YOUR-GITHUB-NAME/clojure-test-lib
Enter fullscreen mode Exit fullscreen mode

After that we'll get the library template:

clojure-test-lib
├── CHANGELOG.md
├── LICENSE
├── README.md
├── build.clj
├── deps.edn
├── doc
│   └── intro.md
├── pom.xml
├── resources
├── src
│   └── YOUR-GITHUB-NAME
│       └── clojure_test_lib.clj
└── test
    └── YOUR-GITHUB-NAME
        └── clojure_test_lib_test.clj
Enter fullscreen mode Exit fullscreen mode

Please take a look at build.clj file that contains available build commands. Note, that the command for creating a JAR file called here ci (which stands for "continuous integration"), because it doesn't only create a JAR but also runs tests.

Also please read the README.md file. It has the examples of the build commands and an important note that you have to fix the existing test to get a successfull build ;)

💖 💪 🙅 🚩
kozieiev
Volodymyr Kozieiev

Posted on October 31, 2021

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Creating and publishing Clojure libraries
clojure Creating and publishing Clojure libraries

October 31, 2021