Game of Phones: History of Erlang and Elixir

How do old technologies become relevant dozens of years after their conception?

Since their origin in telecommunications, Erlang and its VM have gone a long way. But everything that we think of as great was set in motion more than 20 years ago, through the power of permissive design and good decision making.

In this article, we are going to look at the origins of Erlang, how it enables the functional programming paradigm and the actor model.

Together with you, we will also look into the BEAM: the abstract machine that powers Elixir and Erlang, and how it allows software engineers to focus on business logic instead of the computational plumbing.

Finally, we’ll talk about the emergence of Elixir with its cleaner and leaner standard library, easier metaprogramming, and the best web framework known to humankind.

First off, though, a brief venture into computer games.

High-performance entertainment

Video games have always utilized the capabilities of computers to their fullest degree. Take the first popular game: Spacewar. It used the advanced input handling of PDP-1 to allow two players to battle in outer space.

Conversely, computer games also have informed the development of hardware.

Data storage and memory. Setting popular games aside, the necessity to extend the capabilities of computers goes as far as the mid-forties. Then, in a failed attempt to make a flight simulator machine called “Whirlwind”, significant proceedings were made in the field of reliable magnetic data storage.

Co-computing with GPUs. Once computers became more and more alike, the idea of playing video games using them became more appealing. In 1994, Sony released PlayStation featuring a separate processor that was crunching the video rendered in games, coining the term “GPU”.

We didn’t have to wait for long for real-time video processing on “personal computers”. Nvidia released its first GeForce board in 1999. Over time, computer games became more and more demanding. It forced Nvidia and ATI to work hard on parallel processing of video data. Soon, scientists and software engineers realized that not only video data can be crunched on GPUs. For instance, you can contribute to dealing with the 2019 strain of coronavirus, which looks to be the bane of 2020, using your GPU, of course.

Why do we mention computer games, you might think? For the power of analogy. The evolution of BEAM languages has traced the same path: a technology made for high-performance computing in a comparatively niche domain morphs into an industry powerhouse after decades of gestation. Witness.

How reliable phone switching led to scalable startups

1986–1993. Farm to market.

To get to Elixir, we first have to start with its predecessor, Erlang. Therefore, let’s move to the stormy shores of Scandinavia. It’s 1986, and Ericsson is trying to solve the problem of reliable phone switching. The most important success criterion is that such applications have to have zero downtime. In the words of late Joe Armstrong, one of the founding fathers of Erlang, it was a quest to write programs that run forever.

The resulting language is a milestone in programming language development. Erlang is a functional programming language with a focus on concurrency and high availability. It is also the most iconic example of an actor model.

Not unlike computer games, Erlang precedes a lot of approaches and tools that are used in modern computing.

Let’s delve into Erlang’s history and notice those things.

Green threads/fibers. A key design aspect of Erlang as a language is that of a process. It abstracts away operating system processes, bringing the notion to the language level. All the computations in Erlang are done within those abstract processes. Of course, Erlang wasn’t the first language to take this route. However, its design exploits such abstraction to its fullest. Erlang processes are basically green threads.

AMQP. Erlang uses message passing between processes instead of allowing locks over shared resources. It also has so-called mailboxes, which are queues attached to every process. As the process handles the messages, mailboxes get emptied. An overflowing mailbox can be interactively inspected in runtime, enabling great maintenance flexibility. Inspired by experiments with Smalltalk language, it preceded message queue systems. One of the most popular of those is called RabbitMQ and is, incidentally, written in Erlang. Erlang’s message passing is basically AMQP.

Continuous delivery. Telephony applications have to keep running. Thus, the design of Erlang also provides developers with a way to update configurations and even modules live. Erlang code update is basically continuous delivery.

Functional programming. Finally, Erlang is compatible by construction with functional programming. As a matter of fact, Erlang also took a lot of inspiration from logical programming. One can feel it in its “weird syntax”. The first implementation of Erlang was a meta-interpreter written in Prolog. Therefore, Erlang is basically a functional programming language.

All these properties were great from the programming language theory standpoint, but there was a problem – the systems run via Prolog meta-interpreters were painfully slow. To address this performance issue, a virtual machine called JAM (Joe’s Abstract Machine) was introduced and implemented in C by Mike Williams. It was good enough for industrial prototypes, but performance was still an issue.

1993–1998. Heavy industrial use and a path to open source.

Fortunately, a beam of hope for wider industrial usage came from the replacement of JAM with the BEAM (or, as it was known at the start – Turbo Erlang) by Bogumil Hausman in 1993. It was a highly efficient, threaded abstract machine that is a direct predecessor of the modern Erlang VM, which is also – albeit somewhat non-imaginatively – called the BEAM.

Other than the improvement in speed, continuous work on Erlang managed to yield other benefits as well:

Concurrent, parallel, and distributed computing. Concurrently (no pun intended), distributed Erlang was developed, allowing for first-class clusterization of programs and multi-threaded reduction-based scheduling. As long as there are no Byzantine actors, software compiled to the BEAM can be run across multiple computers without the need for OS configuration. Distributed Erlang is basically Kubernetes and Apache Spark in one.

Scalability. Together with zero downtime configuration updates, distributed Erlang allows for transparent scalability in both directions! It can automatically up-scale and down-scale your system. You can achieve both vertical scalability by feeding more cores to the scheduler and horizontal scalability by adding more servers to a cluster. A perfect property for a startup, isn’t it? Erlang is basically AWS Autoscaling.

Open Telecom Platform

In addition to astonishingly expressive first-class abstractions in the language, Erlang also has excellent facilities for handling failure. One such facility, introduced in 1998, is Open Telecom Platform.

The name might be confusing at first, but it actually is just a part of Erlang’s standard library that was used for years to ensure the fault tolerance of telecom systems. OTP gives the user building blocks for arranging processes into a supervised hierarchy. With it, one can define a failure mode in which processes should cause restarts of parts of the system. This way, predicted failure is always contained and the system keeps running. Cascading failure can either be contained by a subsystem restart, based on how the OTP hierarchy is structured in the application. No failure should prevent the system as a whole from being in an operating state.

Aside from that, once a failure is detected, it is possible to remotely attach to any node and inspect any process, even visually, as the system runs!

1998–2011. Adoption of Erlang.

In 1998, Erlang (for reasons, such as the difficulty of maintaining a proprietary language) was banned for new product development at Ericsson Radio. This led to the creation of Open Source Erlang. While one may initially view it as a bad thing, it did ultimately contribute to the spread of Erlang outside telephony. Hooray for open source!

In contrast to other languages, Erlang at the time had significant advantages in concurrency and availability guarantees. Let’s see where one can benefit from using Erlang:

What is Erlang used for?

Anything that involves messaging and distributed/parallel computing is a good bet for Erlang. To see world-class companies that use Erlang, check out our examples of Erlang companies.

• Chat apps. Famous messaging apps such as WeChat and WhatsApp use Erlang, both of which have reported hitting 1 billion daily active users.
• Distributed, high-performance services. If you need to process a lot of transactions coming from a ton of places in your fintech project or create a bidding/user matching platform, Erlang is not the worst choice.
• Message queue systems. Since we mentioned Erlang being basically a proto-AMQP, we have now come full circle. RabbitMQ, an open-source message broker that implements AMQP and other protocols, is a huge success story for Erlang.

Obviously, Erlang is still used in telecommunications. Ericsson, in particular, decided they can’t do without it and it has been adopted by other companies like Cisco.

2011–now. Elixir.

Erlang might be a great language, but it is also one of the most dreaded, according to a Stack Overflow survey. Do you know which one is one of the most loved? Elixir.

When José Valim released an early version of Elixir, it felt like a breath of fresh air. In contrast to Erlang, Elixir had a promise for a more streamlined programming experience that would be more convenient for a modern developer that has used languages like Ruby and Java. But in contrast to them, it would build on the basis of Erlang VM and all the great things it entails.

Elixir delivered, for several reasons.

With its standard library, Elixir ships some handy data structures. It also eliminates boilerplate when it comes to declaring OTP hierarchies and other commonly performed standard function calls.

Elixir has very powerful and safe metaprogramming. Thanks to the quote/unquote functionality, it’s first-class, as opposed to tricky metaprogramming in Erlang. Some practical and ingenious usage of Elixir’s metaprogramming can be seen in the unicode.ex module of Elixir’s standard library.

Finally, Phoenix is hands down the best web development framework that is out there. It offers the convenience of your regular framework while having the powerful BEAM to back it up. (In our list of future blog posts there is one on making a PWA with Phoenix, so stay tuned.)

These days, Elixir is a feature-complete, stable language that respects backward compatibility.

In the words of a great man, “Elixir” is basically “Tooling”.

Conclusion

Not unlike games, the world of telecommunications has given us technology that has bloomed while enabling reliable, multi-user high load systems. For example, WhatsApp built its entire startup on Erlang and FreeBSD.

Among other companies that have successfully used BEAM languages to create large-scale systems that work reliably and serve large numbers of users every day, we can name Facebook, Pinterest, Klarna, Discord, and Grindr. As Alice (in Wonderland) might attest, using an elixir is the best solution to any scaling issue.

Seeing as computer games were one of the main factors pushing the technology forward (because who doesn’t like to entertain themselves), they also probably indirectly contributed to the possibility of BEAM languages jumping from telephony to conventional computers.

These days, BEAM languages are actively contributing to the development of them. For example, World of Tanks has an Erlang implementation of chat and signal management systems. There are companies, like GrandCru, that write entire backends for multiplayer games in Elixir.

And hence, the answer to the question made at the start of the article. If you make something that is useful, solves a certain problem much better than anything else, and take care to make good decisions at every step of the design process, you are likely to come up with a technology that will change the world.

Be like the BEAM. Be useful.

If you want to learn more about BEAM languages and how they can help create scalable solutions for high-volume systems, follow us on Twitter.