Top Open Source functional programming technologies

roman_m

RomanMizulin

Posted on October 29, 2024

Top Open Source functional programming technologies

Author: Roman Matveev. LinkedIn x.com

Purpose

The goals of this article is to research the most popular FP based technologies to prove FP soundness and versatility and to demostrate that FP has been already with us for a long time.
Methodology involved analyzing GitHub for the most popular projects, blog posts and academic papers.

A little bit of historical context

Engaging talk with a lot of references to historical moments emphasizing point that:

"Good" languages are differed from "bad" languages by theoretical base (specifically math logic brought to computer science). Not just engineered.

"Propositions as Types" by Philip Wadler - YouTube

LISP

Ancestor of almost all functional languages.
Originally was designed to solve 1 specific problem of symbolic differentiation, for which FORTRAN was very inconvenient. And eventually became general purpose-purpose language. In that sense (growing from 1 specif problem) it is similar to Erlang.
With time passed became purer language thanks to academic influence.
It had inspired a lot of other languages including the whole family of so-called LISP languages.
The second-oldest high-level programming language still in common use, after FORTRAN.
It brought to the table such essential concepts without which we can not imagine our industry today as :
- Conditionals not limited to goto (if-then-else)
- First-class functions
- Recursion
- Garbage collection
- Dynamic typing
- Interactive programming environment (REPL)
- Full language available at load time, compile time, and run time

Haskell

Appeared in 1990 with stable release at 2010.
If LISP started all funtional obsession in XX century then Haskell summarized ideas of languages appeared in the next 30 years. The main idea was not that academics should not reinvent same features in their languages, but to make common ground for benefits of all. Fun part is that writing I/O related code in Haskell was so annoying, so Monadic I/O was ultimately invented that has groundbraking influence. Here stonk monad plot showing Haskell m-wording the world.

Haskell monading the world

Other

There are plenty languages worth to mention and their role in history, but let me focus on some of them relevant today.

OCaml

Direct descendant of ML which Appeared as dissatisfaction with LISP writing proofers. The idea was to have types which lower logical errors. (LISP -> ML -> OCaml)
OCaml was written in 1996 in France. It was inspired by a long line of research into ML, starting in the 1960s, and continues to have deep links to the academic community.
Less pure comparing to Haskell but more focused on Industry.
So if you want to participate in making tooling for whole ecosystem for future generations - it is a good opportunity to learn and make huge impact.

Erlang

Appered from need for faster develompent to reliable run on telephony switches, because in 80s managers realized that they spend more on developing and maintaning software than on hardware itself. It was designed to be concurent, distributed and high available from beginning long time ago before async fashion in JS and Python. In contrast, OTP and Erlang almost 30 years(open-sourced) old battle tested technology. Interesing, that originally it was propriate closed language of Ericsson's company, which is powering world networking now.

Top projects grouped by language

Elixir

Appeared at 2012. Built on top of BEAM to make Erlang\OTP runtime more accessible, because Erlang is not user friendly really; and shares the same abstractions and OTP framework for building distributed, fault-tolerant applications. Though initially designed with dynamic typing, Elixir is actively developing static typing capabilities cnrs. As Joe Armstrong said about types "no good type system save you from node failure" — that is why Erlang language was dynamically typed — it was not focus. And it is one of the two languages in this list available on Leetcode, but I would recommend to use https://exercism.org/ for practice firstly. You can think about this language as more modern and easy to use language for BEAM runtime than original Erlang language.
Why does it matter? Because Erlang really powers high load. You can find out more here about real world use cases of Erlang its VM called BEAM. Whats-app cloud handles 2 millions connection per a node for instance.
It is simple language nevertheless a powerful one. Maybe Elixir has not really good theoretical fundament, but it is still good language and its author José Valim actively works on bringing type system with synergy of academic world.

Image description
Picture from talk by creator of the lang José about different levels of abstractions Keynote: Gang of None? Design Patterns in Elixir - José Valim | ElixirConf EU 2024 - YouTube

Phoenix: Top from stack overflow surveys

Stars: 21k
One of the most if not the most popular FP frameworks that has won "the most admired web framework" at stackoverflow research for several years in a row.
Stack Overflow Developer Survey 2023
Fullstack, Ruby inspired. Connects high load with easy of use.
There are big companies using this framework check related block at Phoenix Framework

analytics

Stars: 19.1k
Simple, open source, lightweight (< 1 KB) and privacy-friendly web analytics alternative to Google Analytics. Serious niche and good moral idea.
All we can do - is to embrace such kind of projects.

We are dedicated to making web analytics more privacy-friendly.

supabase/realtime

Stars: 6.6k
One of the projects of famous supabase project which leverages Phoenix from the first place. It makes use from BEAM through Elixir.

This is a server built with Elixir using the that enables the following functionality:

  • Broadcast: Send ephemeral messages from client to clients with low latency.
  • Presence: Track and synchronize shared state between clients.
  • Postgres Changes: Listen to Postgres database changes and send them to authorized clients.

Erlang/OTP

Not just language but framework too running on BEAM. Erlang is not user friendly or looks familiar to popular languages so Elixir was created to make this incredebly concurrent platform accesable to regular web developers. .
Joe Armstrong remarked in an interview with Rackspace in 2013:

"If Java is 'write once, run anywhere', then Erlang is 'write once, run forever'."

CouchDB

Stars: 6.2k

If there’s one word to describe CouchDB, it is relax

Document based reliable scalable database with nicely designed HTTP/JSON interface.
With accompanient of Pouchdb can be the best choice for offline-first applications with low effort data syncronisation.

RabbitMQ

Stars: 12.2k

RabbitMQ is a reliable and mature messaging and streaming broker, which is easy to deploy on cloud environments, on-premises, and on your local machine. It is currently used by millions worldwide.

Famious publish/consumer message queue with a lot of scenarios to implement.

emqx

Stars: 14k

The most scalable open-source MQTT broker for IoT, IIoT, and connected vehicles

Gleam

Appeared at 2016. V1 was released at 4 Mar 2024, so it does not have popular projects yet, but it grows fast. Running on Erlang VM as Elixir, but statically typed in opposite of dynamically typed Elixir and Erlang. Inspired by many languages and as author says "I stole the best ideas for this language." © Louis Pilfold .
And I have to admit that coding in Gleam is pleasant experience. It is really easy to learn and fast LS gives instant feedback on type coherence.
If you can choose among Erlang, Elixir and Gleam. You should deferentially look at Gleam, but if you like Ruby look at Elixir of course.
Maybe it is true opensource sponsored language, in this meaning it is a good example of growing contrasting to Elm.
Try it at The Gleam Language Tour

P.S. Remember Golang mascot? yeah... Now look at this piece of art.

Gleam mascot

Front end in Gleam

Lustre allows to write fullstack applications in Gleam reusing same types!
Basically it follows the same MVC architecture as Elm.

Clojure

Clojure is a compiled language, yet remains completely dynamic. Clojure provides easy access to the Java frameworks, with optional type hints and type inference, to ensure that calls to Java can avoid reflection... is a dialect of Lisp, and shares with Lisp the code-as-data philosophy and a powerful macro system.

Appeared at 2007. From quote we can conclude that one of the main factor becoming popular is running on JVM. Concurent of Scala.
Used at Walmart Labs
“Our Clojure system just handled its first Walmart black Friday and came out without a scratch.”
For me dynamic nature is red flag nowdays, especially when there such beatiful languages such Elm, Ocaml, Haskell and maybe Gleam if it woud not have so many braces.
Here are really titans of Clojure world in terms of numbers of stars.

metabase

Stars: 37.4k
Very popular tool for work with data for non-technical people. It let to get and visualize simple data without SQL, but for more complex queries SQL is required. One technical person can make views and all non-technical part of a team can use it.

logseq

Stars: 31.1k
A privacy-first, open-source platform for knowledge management and collaboration.
Notion and Obsidian like alternative.

penpot

Stars: 29.5k
The open-source tool for design and code collaboration.
Figma alternative.
By number of stars we can assume that IT world at least admired this tool.

Elm

Appeared at 2012. Extremely efficient and deligtful to use to build browser UI. Not popular now - in its afterhype period of time, but definitaly impactful. Can be seen as more simple and compact child of Haskell and OCaml. Has elegant type system. Can be learn in a weekend.
There are elm-ui, elm-review popular libraries. And there is not much activity on official site actually, but languages is alive. In talk Evan told that he made compiler from Elm to SQL which sounds cool, but he really struggles to make sustainable financial model. Also seems like Evan does not like online marketing his language still loving conferences. Personnaly I agree with him and I think much people prefer real events to virtual ones but it is important lesson for language designers: online presense matters.
Check Elmcraft and oficial forum for latest news and inspiration.

Haskell

The most academic language in the list. From the beginning made with good type design comparing to the most other popular languages where types were introduced on the go.
Highly admired by creator of ROC language - Richard Feldman. He also praises Elm by the way.
This language has a lot of concepts, so it is a good idea to learn language to explore FP ideas in practice.
And I would say that language is excellent for well understood problem that can be something formally described to leverage its strong type system.
Some researches say that there are better language than Haskell like Agda or Idris. Ok, but for many of FP languages Haskell is a progenitor.
The most elegant and concise language in the list, nevertheless it can be unpleasant to use in production due to various ways to do the same thing.

ShellCheck

Stars: 35.6k
A static analysis tool for shell scripts adopted by:
Travis CI, Codacy, Code Climate, Code Factor, CircleCI via the ShellCheck Orb
GitHub (only Linux), Trunk Check (universal linter; allows you to explicitly version your shellcheck install) via the shellcheck plugin
And you can use it for too if you work with bash/sh scripts.

pandoc

Stars: 33.3k
Universal markup converter. Supports numerous format from/to. Practical and helpful.

PostgREST

Stars: 22.7k
REST API with automatically generated OpenAPI for any PostgreSQL database. Expose database schema with filters and make PostgreSQL views for complex API endpoints.
If you often write CRUD API for database tables manually, it will save you some time.

And more less popular projects

Like hadolint: Dockerfile linter, validate inline bash, written in Haskell; Purescript, Unison, Idris languages implemented in Haskell.
I recommend to check ideas behinds those projects. Especially Unison's mind-blowing something naive idea about infinite computational resources.

Scala

Brings FP to Java land. Good path for whom looking for functional language to code and the best job opportunities. But not elegant language at all in my humble opinion.

spark

Stars: 40k

Apache Spark™ is a multi-language engine for executing data engineering, data science, and machine learning on single-node machines or clusters.

Created in 2009 at UC Berkeley's AMPLab, open-sourced in 2010 - quite some time. De facto standard for big data processing making use of Scala's functionality. Adopted by Netflix, Uber, Pinterest, Alibaba.

Enso

Stars: 7k
Interesting analytical tool and language with one-to-one visual counterpart. Its IDE has some bugs, and requiring login for some reason, but it is worth checking.
And can be used for "Detecting Credit Card Fraud with Enso" as stated in their blog post.

Python with returns lib

Python is not functional language itself. And as I can relate does not want to be: look at lambda for instance. But this package allows to write more robust applications.
Link to lib: GitHub - dry-python/returns: Make your functions return something meaningful, typed, and safe!

In closing

It is safe to say that functional language can be used in almost any domain excluding maybe system engineering, if not counting Rust as functional language.
You may wonder after all why then functional programming languages are not dominating everywhere if it is so good? So Richard gave a good answer to it: Why Isn't Functional Programming the Norm? – Richard Feldman - YouTube. Let it brew.

Do you want full stack type safe scalable app?

Look closer at Gleam or Phoenix. It has good balance of simplicity and FP feature along side with good development experience. Maybe add CouchDB since it runs on BEAM too.
Great article on using Gleam for a long time.

Do you want a queue?

RabbitMQ for general use, EMQX for IoT.

Do you want analytics?

Give Enso or MetaBase a shot.

Do you want complex reliable backend for WEB or for CLI?

Look at Haskell or OCaml. Which one specifically? There is a good article comparing from an experienced developer in both languages.

Do you want frontend?

Elm or Elm inspired frameworks. Fortunately there are infinitely many Elm inspired frameworks like mentioned Lustre or something esoteric like Unison lang library.

Do you want to try FP in TS or Python?

fp-ts for TypeScript, returns for Python.

Do you want insane Data/ML stuff?

Spark

P.S. Radar version of this article would be good.

References

💖 💪 🙅 🚩
roman_m
RomanMizulin

Posted on October 29, 2024

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related