The Singleton Design Pattern

Context

I've recently been refactoring some of the major components of a project I've been working on for the last few years, Proxima. It's a tool to encode proxies in parallel on multiple machines for DaVinci Resolve. I started this with a very entry-level knowledge of Python and had to bend the rules a couple of times and use code I didn't understand to get the behaviour I wanted. One example of this was using singletons. I recently came across a video that discouraged the use of singletons and was intrigued to know more about them. So I dug up my old code to try and figure it out.

NOTE : This is me "learning in public". I research a topic I'm interested in and try to learn more by explaining. Please take everything you read here with a grain of salt; click the links or search for yourself, but don't take my word for it. I am by no means an expert! If you are, please comment and feel free to school me, I'm always keen to learn more.

What is it?

The singleton design-pattern allows a traditional class the super-power of global-state application-wide. Every instantiation returns the same instance under the hood. Imagine a database that's expensive to connect to. You may want to limit your connections to it to just one. Singletons make that easy. For my project, I had a complicated configuration file that needed to be parsed and validated before it could be used. I didn't want that happening every time I used it.

There are a few ways to construct a singleton class in Python. Here's one using a metaclass implementation:

class Singleton(type):
    _instances = {}
    def __call__ (cls, *args, **kwargs):
        if cls not in cls._instances:
            cls._instances[cls] = super(Singleton, cls). __call__ (*args, **kwargs)
        return cls._instances[cls]

class Cake(metaclass=Singleton):
    def __init__ (self):
        # combine dry ingredients
        # combine wet ingredients
        # bake!
        pass

A metaclass is essentially a class of a class, and can be used to define the instantiation of its subclasses. When a class is called and has elected Singleton as its metaclass, Singleton defines that class's instantiation. In this case, if a Cake instance already exists in_instances, It is returned to the caller.

Read up more on metaclasses here: https://realpython.com/python-metaclasses/

and see this SO thread on creating Python singletons : http://stackoverflow.com/questions/6760685/ddg#6798042

When should I use it?

Generally good candidates for the Singleton pattern are:

Immutable data sources with slow/expensive parsing (data that does not mutate cannot become inconsistent)
Data "sinks" like loggers or console interfaces (data does not affect internal application state and logger configuration can be made consistent without configuring every instance)

Why is it considered an Anti-Pattern?

Singletons can cause issues when their global state is abused. If you need the same data and functionality in multiple places, singletons make that easy. If that data is complex, mutable data, the potential for state issues becomes much higher.

In essence, singletons are considered an anti-pattern in modern development because they:

have implicit global-state
are difficult to unit test
are difficult to use with multi-threading
often break the "single responsibility" rule

Overwhelming internet opinion is to avoid them.

See below for others opinions:

A Brief Rant on Anti-Patterns

Human beings like extremes. They don't like grey areas. We gravitate towards different ends of the spectrum. I believe one of these black and white trends is badging things as "anti-patterns". "Don't use/do ___ it's an anti-pattern." 'Nuff said. I don't think it's wise to see such things so black and white. Context is key to revealing grey areas; you alone know your code and use case; you alone know how the potential pros and cons will affect you. I believe good use of an "anti-pattern" is possible if you can avoid the cons and you can't get the pros in a better way. Judge it by its fruits; how it affects your code. Not someone else's popular opinion.

Why I started using Singletons in Python

My programming journey started with me bouncing around some of the less syntactically favoured languages looking for quick macros snippets and automations at work and just getting them to barely do what I need. I didn't really want to learn anything, I just wanted shortcuts. But I grew to love it. I've since had to become more patient, but some bad habits remain and unfortunately I missed some of the basics. "Classes" was one of those basics. When I started getting into Python and developing more complex automations I realised I was reaching limits playing object/function ping-pong. I had come so far without them, I was pretty hesitant in starting. I even considered functional programming first. Eventually I acknowledged my denial and agreed I needed to learn the rules before I could forego them. So I took to Python classes 101... sort of.

My Configuration Manager use case

My first class was not a "hello world" class and it really should've been. It was a configuration manager. And like every aspiring programmer I wrote my own thinking that there was no configuration manager out there that would "suit my needs", so I built my monstrosity and overloaded it with features like:

Parse YAML files as neat and tidy, human-readable configuration
Counteract YAML's easy going nature with schema-based type validation
Detect missing configuration file and prompt user to modify copied defaults
Surgically insert updates while retaining YAML comments, order and nesting (yikes)
Check for keys missing from the configuration and prompt user for subsitution with default or custom values
Warn user about stray unsupported keys

At the time, I figured I was pretty hot stuff now that I had neatly organised classes instead of functions grouped in comment fences. My configuration manager was working, so I imported it into all my modules that needed access to the configuration object and it worked! Well, mostly.

Slow Initialisation

Unfortunately it turned out my new class wasn't going to win any benchmarks. It wasn't just parsing YAML on each init that was slowing things down, it was all the data validation gymnastics. Parsing and validation running on application start is necessary, but reinitialising on every import of the settings object is not... and there were very few modules that didn't import the configuration, since the application logging level was configured in YAML. Granted, I could probably have written things for more effective performance...

Not only was it slow, but I had already added cool, coloured log messages, so I had "Checking settings..." pop up for a few seconds every time execution context changed. It was like my own program was shaming me. This is yet another thing I struggle to stick to: prioritising function over form; I get carried away with details before I have the MVP. Eat your vegetables first, dessert later.

I gave myself a little bit of grace, since I was exploring the inner-city of OOP for the first time, and looked for a solution. Pretty soon I stumbled upon singletons and had no idea how they worked, so yeet -> delete went my old code and yank -> thank went me on Stack Overflow. Problem solved.

In short, using a singleton allowed my class to skip running __init__ for every instance. On paper, this was exactly what I needed. In fact a slow, read-only configuration manager is one of the better use cases for singletons. But I know in future I'm likely to have cross-module setting updates and need for a solid unit-testing pipeline. Those are pretty high on the list of cons for traditonal singletons. Plus I'm all in for Python at the moment and I'd like to learn the recommended way.

The "Most Pythonic" Way

While it is possible to create and use traditional singletons in Python, more "Pythonic" alternatives exist. If your entire team consists of first-time python-using Java developers creating a one off app that needs no unit testing, using traditional singletons in Python might not be so bad.

There are two "pythonic" ways of doing this. The latter is just a variation of the former really:

module level code
package level code

If you're hazy on the difference between scripts, modules and packages, take a look here: https://realpython.com/lessons/scripts-modules-packages-and-libraries/

Module-style "Singleton"

Here's a script. Notice everything takes place in the one file. We get our cake just by running it.

# bake_cake.py script

def combine_dry_ingredients(dry:List):
    return "-".join(dry)

def combine_wet_ingredients(wet:List):
    return "-".join(wet)

def bake(dry:str, wet:str):
    return dry + wet

if __name__ == " __main__":

    dry = ["flour", "baking powder", "cocoa powder"]
    wet = ["eggs", "milk", "vanilla essence"]

    combined_dry = combine_dry_ingredients(dry)
    combined_wet = combine_wet_ingredients(wet)
    baked_cake = bake(combined_dry, combined_wet)

    print(f"Baked cake!\n{baked_cake}")

Here it is as a module. We have initialised all the variables, but we're not using them for anything.

# bake_cake.py

def combine_dry_ingredients(dry:List):
    return "-".join(dry)

def combine_wet_ingredients(wet:List):
    return "-".join(wet)

def bake(dry:str, wet:str):
    return dry + wet

dry = ["flour", "baking powder", "cocoa powder"]
wet = ["eggs", "milk", "vanilla essence"]

combined_dry = combine_dry_ingredients(dry)
combined_wet = combine_wet_ingredients(wet)
baked_cake = bake(combined_dry, combined_wet)

If we import that module, all of those variables are accessible like this :

import cake

print(cake.combined_dry)
print(cake.combined_wet)
print(baked_cake)

There are a couple of downsides here. You might not want to run and cache everything when the module is imported. If the dry and wet ingredients were not hardcoded lists, but instead read from a .csv file or from an API, we'd probably want to be doing that waiting at call time, not import. This is called "lazy loading". There are libraries on PyPi and design patterns you can employ to implement lazy loading at a functional level. If you need a class-based implementation Take a look at the @property decorator.

See Real Python's great tutorial on this.

Package-style "Singleton"

You can take the module approach a little further using packages. The cake example is not entirely realistic, but hopefully it still illustrates the point. This is all really much the same procedure as with modules, but the global variables live in the package __init__.py file instead.

It's a bit of extra set up but you gain:

Greater separation of logic and state
No need for an ambiguous "main" module or function
All the benefits of caching with no enforced single instance
Easy unit testing: just import modules individually instead of the whole package

This makes unit testing trivial. If we want to test individual modules and functions we don't have to contend with enforced single instance or global state, we just import the modules separately instead of the whole package.

# cake.py

def combine_dry_ingredients(dry:List):
    return "-".join(dry)

def combine_wet_ingredients(wet:List):
    return "-".join(wet)

def bake(dry:str, wet:str):
    return dry + wet


# ingredients.py

import os
def get_env_ingredients():
    dry = os.environ.get("CAKE_DRY_INGREDIENTS")
    wet = os.environ.get("CAKE_WET_INGREDIENTS")
    return dry, wet

def get_csv_ingredients():
    pass

def fetch_api_ingredients():
    pass


# __init__.py

import package.cake
import pakage.ingredients

dry, wet = package.get_env_ingredients(dry, wet)
c_dry = package.cake.combine_dry_ingredients(dry)
c_wet = package.cake_combine_wet_ingredients(wet)
package.cake.bake(c_dry, c_wet)

Wrapping Up

Unless you need to enforce a single class instance with metaclassing, using modules or packages to cache data will probably replace any need you have for singletons. Of course before you go fixing what isn't broken, consider the cost of getting familiar with the Python ways and the cost of refactoring. Right-est isn't always best. if you can think of a use case for singletons in Python that stands up to modules and packages, I'd love to hear it! I'm certainly not a master of Python, nor any other language.

Why isn't the Module Approach more Popular?

Unfortunately because Singletons are a pattern that exist in many languages, the Python module approach isn't exactly the top of a Google search when you're trying to describe a re-initialising problem. Those who have used singletons in other languages before coming to Python will likely search "How to do singletons in Python".

Maybe this is all part and parcel of jumping the gun and finding general solutions to general problems. This is where deep diving language courses really pays off.

Some Alternatives

Here are a couple of other alternatives I found along the way. There isn't 1 for 1 feature overlap here, but they seek to solve some of the shortcomings of singletons.

Monostate Pattern

The monostate design pattern tries to solve some of the issues with singletons by allowing multiple instantiations, but sharing static data between those instances behind the scenes.

They are easier to inherit from, modify and unit-test than singletons:

Object Pool

Here's an alternative design pattern that works well when you want to set an instantiation limit larger than one and not deal with implicit global state. This works great as a hard limit to resources. Thanks ArjanCodes! https://youtube.com/watch?v=Rm4JP7JfsKY

Arjan also goes into more detail as to why he considers Singletons an anti-pattern in Python.

Blog

Singleton Design Pattern in Python?

Caleb Trevatt