Lifting the hood of the closures

Originally posted on https://www.celsojr.com/post/closures-lifting-the-hood

Closure is not something new. There are a bunch of articles and tutorials out there on the internet talking about closures. In fact, closures have been around for several years since this term was first introduced by Peter J. Landin in the year 1964 [1], the same year my mother was born.

But thanks to the Scheme programming language (a lexically scoped variant of Lisp) which was defined by Sussman and Steele in 1975, closures then became very popular. After that, pretty much all programming languages have closures.

Scheme is a functional-first programming language. Closures are most commonly used in a functional programming paradigm context. But we can see the use of closures in enterprise applications nowadays because most programming languages, like Scheme, are multi-paradigm even though some of them are not functional-first. Only the most simple/primitive programming languages are single-paradigm.

But despite being an old subject, closures are still a hot topic today. For example, in this report from the Rust programming language, asynchronous closures are one of the most desired features that people hope will be stabilized.

If you are a beginner and you want to better understand this programming concept, I suggest that you first try to understand about scope and compositions. A scope refers to the region of code where a variable or function is accessible. It determines where you can reference and use variables or functions within your program. In C# and many other languages, the simplest scope can be delimited between curly braces. Look at this pseudo code example:

var a = "outer scope";

{
    var b = "inner scope";
    var c = a; // Yeah, 'c' can see the 'a' outer scope from here
}

var d = b; // ERROR: But 'd' cannot see 'b'

This is still not a closure because there is no closed operation/expression involved.

I'm just trying to clarify things so that everybody can understand.

Imagine that you are a function and you ate food a few hours ago. Now you are a closure entering the bathroom with the food variables in your stomach to process your stuff — and please, I don't need to know details about how you process your stuff from here, on the outer scope ;)

When a variable in the outer scope is captured by a function to be part of its enclosed processing block in a different scope or environment resulting in a closed operation or expression, we can call this a closure. But this is not the definition, I'm just trying to explain closures with my own words. You can do quick research and find multiple different explanations better than mine.

But why are closures really necessary? Well, as we are talking about scope, they are necessary because those functions or expressions may want to use the variables in its inner scope regardless of what is happening in the outer scope or not. Also, depending on the API you are working with, it should allow you to create powerful compositions by using closures wisely.

In C#, the closures were introduced to help with the anonymous functions and lambdas in the language version 2.0 [2]. Closures make it much simpler to create delegates and design APIs which use delegates.

For example, the entire Linq [3] is built on top of this concept. In my opinion, Linq is a very well-designed API, although it was not initially designed with major concerns in terms of low-level programming and memory consumption. It allows us to make beautiful compositions to better express our intentions without having to create separate functions throughout the application. Also, most code editors and IDEs should provide you with IntelliSense[4] and/or IntelliCode[5] capabilities to empower your developer experience and productivity.

Both approaches, query or method syntax, are composed using closures because query syntax translates into method syntax at runtime anyway. Most applications should use both approaches together.

Without IntelliSense and IntelliCode, compositions with closures wouldn't be the same. Some time ago, my friend at Microsoft, Gustavo, and I had some (external) conversations via email about IntelliCode and he shared with me a paper [6] about the idea behind the IntelliCode. He is Brazilian too. It was a wonderful read and it's always a great experience to be able to work with IntelliCode in Visual Studio. IntelliCode is an amazing tool from Gustavo, Mark and their team.

Closures capture

Programmers working with Linq make use of closures all the time, even if they are not aware of it. Closures are normally captured by reference. And in C#, this happens in an implicit way. Let's look at this pseudo code example:

using System;

int num = 0; // outer scope

Action added = (n) =>
{
    num = num + n; // inner scope
};

added(1);

Console.WriteLine(num); // Output will be 1

As you can see, we are using an Action delegate to sum up two numbers. The num variable, which is in the outer scope, is being captured by reference by the closure and the delegate in the inner scope. This way, when the action is executed, the original variable num is changed.

In C# there are a variety of delegate types with different flavors. There is also the local functions [7] that can work as closures if needed.

The language provides flexible APIs that allow us to work with transformations and compositions as we prefer. But we won't go into detail about delegates or local functions for now.

But how to check how the closure is being captured and whether it's being captured by reference or value? Now let's take a look under the hood, at one lower level, at what the compiler is doing for us:

[CompilerGenerated]
internal class Program
{
    [CompilerGenerated]
    private sealed class <>c__DisplayClass0_0 // <- 😐 Compiler generated class
    {
        public int num;

        internal void <<Main>$>b__0(int n)
        {
            num += n;
        }
    }

    private static void <Main>$(string[] args)
    {
        <>c__DisplayClass0_0 <>c__DisplayClass0_ = new <>c__DisplayClass0_0();
        <>c__DisplayClass0_.num = 0;
        Action<int> action = new Action<int>(<>c__DisplayClass0_.<<Main>$>b__0);
        action(1);
        Console.WriteLine(<>c__DisplayClass0_.num);
    }
}

In this code snippet generated by SharpLab, we can see that the compiler has generated some sort of helper class that contains the closure capture, in this case, the variable num. It's worth remembering that a class is a reference type object in C# and, therefore, will be allocated in the managed heap. And every time this delegate Action is invoked, the num variable of the same instance is changed.

If this behavior is not intended and you want to avoid this by any reason, you have to "manually" make a copy of the value yourself because there is no reserved keyword or any other mechanism that does this for you. See how to do it:

using System;

int num = 0; // outer scope

Action added = (n) =>
{
    int closureCapture = num; // Captured by value, not by reference
    closureCapture = closureCapture + n;

    // Do whatever you want with the closure capture copy ...
};

added(1);

Console.WriteLine(num); // Output now will be 0

Very simple, don't you think? Yeah, but closure capture may be done slightly different in other programming languages. For example, I prefer the way PHP allows us to explicitly capture closures with the help of the use keyword and the & sign, as shown:

<?php
    $num = 0;

    // Closure explicitly captured by reference
    $added = function($n) use (&$num) {
        $num = $num + $n;
    };

    $added(1);

    echo $num; // Output will be 1
?>

To me, the fact that a strongly typed language like C# captures closures implicitly is funny. Whereas PHP, which is a weakly typed language, does this explicitly. I mean, of course we can express ourselves however we want in both languages, but at least in PHP there is a reserved word use for that. And, by the way, in PHP it seems like there is a reserved word and a method for almost everything. I don't know if that's good or not. Judge yourselves!

Please check out the next blog post about closures.

Disclaimer

It's worth noting that I'm not a Microsoft employee. All opinions in this blog post are my own. The information displayed here is not endorsed by Microsoft, .Net Foundation or any of their partners. This is not a sponsored post. All rights reserved.

1. Closure (computer programming) Wikipedia, Retrieved April 18, 2024.
2. Richard B., & Andrew C., Pro Asynchronous Programming with .NET, Apress, 2013, p. 23.
3. Language Integrated Query (LINQ) Learn Microsoft, Retrieved April 21, 2024.
4. IntelliSense in Visual Studio Learn Microsoft, Retrieved April 21, 2024.
5. Visual Studio IntelliCode, Retrieved April 21, 2024.
6. Paper (PDF) Anders Miltner, Sumit Gulwani, Vu Le, Alan Leung, Arjun Radhakrishna, Gustavo Soares, Ashish Tiwari, and Abhishek Udupa. 2019. On the Fly Synthesis of Edit Suggestions. Proc. ACM Program. Lang. 3, OOPSLA, Article 143 (October 2019), 29 pages. https://doi.org/10.1145/3360569
7. Local functions (C# Programming Guide) Learn Microsoft, Retrieved April 21, 2024.