Sharing Variables between Languages with Polyglot Notebooks
Matt Eland
Posted on May 12, 2023
While Polyglot Notebooks certainly brings the dream of notebook development to dotnet, Polyglot is at its finest when you work with one language and then hand off data to the next language for additional processing.
In this article we'll talk about sharing variables between kernels using Polyglot Notebooks and VS Code. We'll explore the syntax and tooling that exists around these functionalities as well as the current limitation of sharing variables between kernels.
For simplicity, I'm going to avoid getting into SQL and KQL kernels in this article, but I plan on delving further into each of these specialized kernels in future articles.
Understanding Variables and Kernels
As I've noted before, Polyglot Notebooks are powered by the .NET Interactive kernel, which is actually an amalgam of a number of different language-specific kernels such as the C# kernel and the F# kernel all operating under a larger umbrella.
This means that variables you define in C# will not be immediately available in F# (and vice versal).
For example, given the following C# and F# cells:
int year = 2023;
let month = "May"
It might stand to reason that the C# year
variable would be available in subsequent F# cells as well as C# cells, but the year
variable is not defined in the F# kernel, just as the month
variable is not defined in the C# kernel.
To help visualize this, you can look at the Variables
view in VS Code which shows you what variables currently exist in the Notebook, what their current values are, and which kernel they belong to.
Here we see that year
is an Int32
defined in the C# kernel while month
is a String
defined in the F# kernel.
Sharing Variables between Kernels
Thankfully, we can share variables between our different kernels, so if we did some work in the C# kernel and wanted to hand off the data to the F# kernel for further processing, we could do this.
There's a magic command called set
that allows you to declare a new variable in a different kernel.
Here's the set
command that allows you to create a new variable in the F# kernel based on the current value of the year
variable in the C# kernel:
#!set --value @csharp:year --name year
Note: F# is not mentioned in the code above. It is inferred by Polyglot Notebooks by running this code in an F# code cell
When this code runs, the F# kernel now has its own year
variable with its own unique value that is distinct from the value in the C# kernel, though these two variables will happen to store the same value initially.
I personally have trouble remembering the exact syntax of the #!set
command, so I take advantage of a helpful shortcut. To share any variable in Polyglot Notebooks, you can click on the "share" button at the right of the line in the Variables pane.
Doing so then allows you to determine which language you want to share the variable with.
Once you've selected the target language, a new cell is created at the bottom of your notebook with the appropriate command.
There is no difference between typing in the #!set
code yourself versus using the user interface, but in practice I find using the user interface takes me out of the flow less and allows me to focus more on the task I'm trying to achieve.
Sharing Objects between Languages
Now that we've seen how to share variables between different language kernels, let's talk about sharing objects between languages.
First of all, it is possible to share an object defined in one language kernel with another language kernel, but it does not work the way you might expect.
Sharing an object from one kernel to another works via the #!set
command in the exact same way you'd use it with a normal variable, however the sharing process works differently.
When a complex object is shared between kernels, the object is serialized to JSON. That JSON is then represented as a JsonDocument
that is then shared to the other language kernel.
Getting access to data in a JsonDocument
is a bit more cumbersome than working with an object, but can be done by using the GetProperty
method on the RootElement
property as shown below:
let x = p.RootElement.GetProperty("X").GetInt32()
let y = p.RootElement.GetProperty("Y").GetInt32()
Personally, I find this a bit awkward, but it does work and can help you build applications that use multiple languages to carry out a task.
One thing that's very important about sharing objects between languages is that any methods or other forms of logic associated with the object in the original kernel are not transferred over. This is because Polyglot is built on the idea of passing data between different languages for processing, not passing the same objects around with their shared logic.
If you want to be able to share logic between a C# app and an F# app, the dotnet common language runtime already supports this type of workflow in Visual Studio using class libraries and project references. However, this is not the central focus of Polyglot Notebooks and you'll have a happier time working with multiple languages when you accept that Polyglot is instead oriented around handing data off between multiple languages, not sharing a single common language runtime environment across all supported programming languages.
Next Steps
I'd encourage you to play around with how variable sharing works in Polyglot Notebooks as this helps reinforce the idea of multiple kernels operating side by side. While there's a number of things you can't do with variable sharing, you'll find that once you get into the mindset of chaining operations from one language to another that these obstacles are not as large as you imagine them to be.
Ultimately, Polyglot Notebooks is a powerful editing environment that lets you choose whatever language you want for the task you're trying to accomplish. Learning how variable sharing works will help you significantly during adoption and is absolutely worth doing early on.
Posted on May 12, 2023
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.