Demystification of Macros in Nim
Jason Beetham
Posted on September 4, 2020
At first glance Nim’s metaprogramming certainly may drive you to scream and run away from your computer to the nearest ditch. Luckily though macros are not overly complicated, they are just a way for programmers to write code with code so we can generate logic and behaviour programatically. This write-up will document creating macros.
Implementing Go’s Walrus Operator
Go has an operator which defines a variable and sets the value without having a keyword. The following is what it looks like in Go.
a := 300
This can be done rather simply in Nim, but first let’s look at what we need using dumpTree
.
import macros
dumpTree:
var a = "Test"
The compiler will output anything below dumpTree
, so now we can see the AST required for that code.
StmtList
VarSection
IdentDefs
Ident "a"
Empty
StrLit "Test"
So if we want to automate this we see that we need the Varsection, an ident, and a value. So let's look at two ways of doing this.
Method #1
import macros
macro `:=`(name, value: untyped): untyped = newVarStmt(name, value)
That's it, we did it, to be certain though we can use the .repr
procedure to check.
import macros
macro `:=`(name, value: untyped): untyped =
result = newVarStmt(name, value)
echo result.repr
a := "Test"
Running the above code we will see the compiler echo out the desired var a = "Test"
Method #2
This method is equally as short, but uses a nice tool. quote do
lets you write code and it will generate the AST for you, so it's as simple as below. The same checks can be done like above, but I will skip them for the sake of reducing redundancy, as being redundant and repeating yourself is redundant.
import macros
macro `:=`(name, value: untyped): untyped =
quote do:
var `name` = `value`
a := "Test"
Making a Compact If Statement
Sometimes you will want to change the behaviour or function of the language, although Nim does not let you change the syntax you can make your own block logic. One case of this is making a custom if
statement. Below is a desired implementation
let a = "Yellow"
expandIf:
a == "Hello": echo "Good bye"
a == "Yellow":
echo "Would be a lot cooler if you liked blue."
echo "Yellow sucks"
_: echo "You did not speak."
The first condition will be if
, the following conditions will be elif
and finally _
will be used for the else
branch. This means no repeated keywords in our for fun implementation!
import macros
dumpTree:
if a == "Hello": echo "Good bye."
elif a == "Yellow":
echo "Would be a lot cooler if you liked blue."
echo "Yellow sucks."
else: echo "You did not speak."
The above AST looks like such.
StmtList
IfStmt
ElifBranch
Infix
Ident "=="
Ident "a"
StrLit "Hello"
StmtList
Command
Ident "echo"
StrLit "Good bye."
ElifBranch
Infix
Ident "=="
Ident "a"
StrLit "Yellow"
StmtList
Command
Ident "echo"
StrLit "Would be a lot cooler if you liked blue."
Command
Ident "echo"
StrLit "Yellow sucks."
Else
StmtList
Command
Ident "echo"
StrLit "You did not speak."
We can also dumptree the body of our original expandIf
idea, and see the resulting AST nodes we can sample from.
StmtList
Infix
Ident "=="
Ident "a"
StrLit "Hello"
StmtList
Command
Ident "echo"
StrLit "Good bye"
Infix
Ident "=="
Ident "a"
StrLit "Yellow"
StmtList
Command
Ident "echo"
StrLit "Would be a lot cooler if you liked blue."
Command
Ident "echo"
StrLit "Yellow sucks"
Call
Ident "_"
StmtList
Command
Ident "echo"
StrLit "You did not speak."
Looking at both of them you can see in the expandIf
body we can get the required infixes for the ElseIfBranch node. If we can seperate the StmtList from each infix, we then can use the newIfStmt
macro to generate the elif branch using cond
which is the infix and body
which is the StmtList.
import macros
macro expandIf(statement: untyped): untyped=
var
branches: seq[(NimNode, NimNode)] #Condition, Body
elseBody: NimNode #Else code
for cond in statement:
#Based off the dumpTree, we know this is the else body.
let ifBody = cond.findChild(it.kind == nnkStmtList)
if cond.kind == nnkInfix:
cond.del 3 #Removes Stmtlist
branches.add((cond, ifBody))
elif cond.kind == nnkCall and $cond[0] == "_":
#Based off the dumpTree, we know this is the else body.
elseBody = ifBody
result = newIfStmt(branches) #Generates if stmt
result.add newNimNode(nnkElse).add(elseBody) #Appends else body
echo result.repr
expandIf:
11 == 13 : echo "Test"
12 == 14: echo "Huh"
_ : echo "duh"
In the above code you can see the extraction using findChild
. Removal of the StmtList with the cond.del 3
, and finally the creation of the if
. When we compile the compiler sends us a nice message thanks to the echo result.repr
, which is exactly what we wanted a fully constructed if/else statement.
if 11 == 13:
echo "Test"
elif 12 == 14:
echo "Huh"
else:
echo "duh"
Untyped vs. typed macros
So far we have only looked at the usage of untyped macros, these macros do not have any type information and are parsed then passed to our macro. The other type of macro is a typed macro, these are semantically checked which means they have type information and the code has to be valid. Typed macros are useful for anything that requires introspection.
Our First typed macro
Give me a symbol and I will show you the world. In Nim macros symbols or "syms" are the magic sauce to make our spaghetti, they are semantically checked which means they internally store the type they are. The power this gives is all powerful. The first typed macro we will make is one that resets a variable to it's declared value.
The following is the usage:
var a = 100
a *= 3
assert a == 300
resetToDecl(a)
assert a == 100
To start off we know we need a single typed
parameter so our macro header is macro resetToDecl(val: typed): untyped
. typed
is much like untyped
in that it takes any code, but in this case we know it's semantically checked. So to ensure we are only dealing with a variable we must remove all other kinds filtering nnkSym
. Using the nifty error
tool to give a helpful message, which takes in a string and an optional NimNode as a second parameter to provide the line information for the error.
macro resetToDecl*(val: typed): untyped =
if val.kind == nnkSym:
# We will implement code here later
else:
error("This macro only works with variables." val)
Now though we need to filter out all non-var symbols, since symbols can be any type, proc, variable, ... anything you can know by name. So we'll use the symKind
of the symbol to filter out everything but var.
macro resetToDecl*(val: typed): untyped =
if val.kind == nnkSym and val.symKind == nskVar:
## We'll continue here in next block
else:
error("This macro only works on variables", val)
We have done it, we just have variables, but now the question is "How do we get the declaration statement?!", through the power of introspection magic! Nim's macros
module exposes multiple other avenues of introspection other than symKind
it also has getImpl
, getTypeImpl
, getType
, and so much more. The procedure we will use is of course getImpl
cause we want to get the implementation of the symbol(Yes having a symbol is required, batteries not included otherwise).
We can quickly just test to see what the AST is like simply with echo val.getImpl
, what we see is a lovely declaration.
IdentDefs
Sym "a"
Empty
IntLit 100
If you squint closely you can see this is quit a predictable situation, all we have to do is assign the val
to the val.getImpl[^1]
so let us do that!
macro resetToDecl*(val: typed): untyped =
if val.kind = nnkSym and val.symKind == nskVar:
result = nnkAsgn.newTree(val, val.getImpl[^1])
else:
error("This macro only works with variables", val)
I already hear it "Hey is that it, that was easy, there has to be a catch?" and you're right, simply doing var a: int
has caused use to get an error that says Error: illformed AST:
. The reason for this error is that the macro does not check if the last entry in the symbol is nnkEmpty
so all that needs to be done is to make it so if the last entry is empty we emit a default(typeof(val))
.
Let us do that:
import std/macros
macro resetToDecl*(val: typed): untyped =
if val.kind == nnkSym and val.symKind == nskVar:
let impl = val.getImpl
result = nnkAsgn.newTree(val):
if impl[^1].kind == nnkEmpty:
newCall("default", newCall("typeOf", val))
else:
impl[^1]
else:
error("This macro only works on variables", val)
Now to test we've done it properly:
var a = 100
a *= 3
assert a == 300
resetToDecl(a)
assert a == 100
var b: int
b = 300
assert b == 300
b *= 3
assert b == 900
b.resetToDecl
assert b == 0
If you want to see examples I have a variety of packages with mixed complexity of macros:
Posted on September 4, 2020
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.