Demystification of Macros in Nim

beef331

Jason Beetham

Posted on September 4, 2020

Demystification of Macros in Nim

At first glance Nim’s metaprogramming certainly may drive you to scream and run away from your computer to the nearest ditch. Luckily though macros are not overly complicated, they are just a way for programmers to write code with code so we can generate logic and behaviour programatically. This write-up will document creating macros.

Implementing Go’s Walrus Operator

Go has an operator which defines a variable and sets the value without having a keyword. The following is what it looks like in Go.

a := 300
Enter fullscreen mode Exit fullscreen mode

This can be done rather simply in Nim, but first let’s look at what we need using dumpTree.

import macros
dumpTree:
  var a = "Test"
Enter fullscreen mode Exit fullscreen mode

The compiler will output anything below dumpTree, so now we can see the AST required for that code.

StmtList
  VarSection
    IdentDefs
      Ident "a"
      Empty
      StrLit "Test"
Enter fullscreen mode Exit fullscreen mode

So if we want to automate this we see that we need the Varsection, an ident, and a value. So let's look at two ways of doing this.

Method #1

import macros
macro `:=`(name, value: untyped): untyped = newVarStmt(name, value)
Enter fullscreen mode Exit fullscreen mode

That's it, we did it, to be certain though we can use the .repr procedure to check.

import macros
macro `:=`(name, value: untyped): untyped = 
  result = newVarStmt(name, value)
  echo result.repr

a := "Test"
Enter fullscreen mode Exit fullscreen mode

Running the above code we will see the compiler echo out the desired var a = "Test"

Method #2

This method is equally as short, but uses a nice tool. quote do lets you write code and it will generate the AST for you, so it's as simple as below. The same checks can be done like above, but I will skip them for the sake of reducing redundancy, as being redundant and repeating yourself is redundant.

import macros
macro `:=`(name, value: untyped): untyped = 
  quote do:
    var `name` = `value`
a := "Test"
Enter fullscreen mode Exit fullscreen mode

Making a Compact If Statement

Sometimes you will want to change the behaviour or function of the language, although Nim does not let you change the syntax you can make your own block logic. One case of this is making a custom if statement. Below is a desired implementation

let a = "Yellow"
expandIf:
  a == "Hello": echo "Good bye"
  a == "Yellow":
    echo "Would be a lot cooler if you liked blue."
    echo "Yellow sucks"
  _: echo "You did not speak."
Enter fullscreen mode Exit fullscreen mode

The first condition will be if, the following conditions will be elif and finally _ will be used for the else branch. This means no repeated keywords in our for fun implementation!

import macros
dumpTree:
  if a == "Hello": echo "Good bye."
  elif a == "Yellow":
    echo "Would be a lot cooler if you liked blue."
    echo "Yellow sucks."
  else: echo "You did not speak."
Enter fullscreen mode Exit fullscreen mode

The above AST looks like such.

StmtList
  IfStmt
    ElifBranch
      Infix
        Ident "=="
        Ident "a"
        StrLit "Hello"
      StmtList
        Command
          Ident "echo"
          StrLit "Good bye."
    ElifBranch
      Infix
        Ident "=="
        Ident "a"
        StrLit "Yellow"
      StmtList
        Command
          Ident "echo"
          StrLit "Would be a lot cooler if you liked blue."
        Command
          Ident "echo"
          StrLit "Yellow sucks."
    Else
      StmtList
        Command
          Ident "echo"
          StrLit "You did not speak."
Enter fullscreen mode Exit fullscreen mode

We can also dumptree the body of our original expandIf idea, and see the resulting AST nodes we can sample from.

StmtList
      Infix
        Ident "=="
        Ident "a"
        StrLit "Hello"
        StmtList
          Command
            Ident "echo"
            StrLit "Good bye"
      Infix
        Ident "=="
        Ident "a"
        StrLit "Yellow"
        StmtList
          Command
            Ident "echo"
            StrLit "Would be a lot cooler if you liked blue."
          Command
            Ident "echo"
            StrLit "Yellow sucks"
      Call
        Ident "_"
        StmtList
          Command
            Ident "echo"
            StrLit "You did not speak."
Enter fullscreen mode Exit fullscreen mode

Looking at both of them you can see in the expandIf body we can get the required infixes for the ElseIfBranch node. If we can seperate the StmtList from each infix, we then can use the newIfStmt macro to generate the elif branch using cond which is the infix and body which is the StmtList.

import macros
macro expandIf(statement: untyped): untyped=
  var
    branches: seq[(NimNode, NimNode)] #Condition, Body
    elseBody: NimNode #Else code
  for cond in statement:
    #Based off the dumpTree, we know this is the else body.
    let ifBody = cond.findChild(it.kind == nnkStmtList)
    if cond.kind == nnkInfix:
      cond.del 3 #Removes Stmtlist
      branches.add((cond, ifBody))
    elif cond.kind == nnkCall and $cond[0] == "_":
      #Based off the dumpTree, we know this is the else body.
      elseBody = ifBody

  result = newIfStmt(branches) #Generates if stmt
  result.add newNimNode(nnkElse).add(elseBody) #Appends else body
  echo result.repr



expandIf:
  11 == 13 : echo "Test"
  12 == 14: echo "Huh"
  _ : echo "duh"
Enter fullscreen mode Exit fullscreen mode

In the above code you can see the extraction using findChild. Removal of the StmtList with the cond.del 3, and finally the creation of the if. When we compile the compiler sends us a nice message thanks to the echo result.repr, which is exactly what we wanted a fully constructed if/else statement.

if 11 == 13:
  echo "Test"
elif 12 == 14:
  echo "Huh"
else:
  echo "duh"
Enter fullscreen mode Exit fullscreen mode

Untyped vs. typed macros

So far we have only looked at the usage of untyped macros, these macros do not have any type information and are parsed then passed to our macro. The other type of macro is a typed macro, these are semantically checked which means they have type information and the code has to be valid. Typed macros are useful for anything that requires introspection.

Our First typed macro

Give me a symbol and I will show you the world. In Nim macros symbols or "syms" are the magic sauce to make our spaghetti, they are semantically checked which means they internally store the type they are. The power this gives is all powerful. The first typed macro we will make is one that resets a variable to it's declared value.

The following is the usage:

var a = 100
a *= 3
assert a == 300
resetToDecl(a)
assert a == 100
Enter fullscreen mode Exit fullscreen mode

To start off we know we need a single typed parameter so our macro header is macro resetToDecl(val: typed): untyped. typed is much like untyped in that it takes any code, but in this case we know it's semantically checked. So to ensure we are only dealing with a variable we must remove all other kinds filtering nnkSym. Using the nifty error tool to give a helpful message, which takes in a string and an optional NimNode as a second parameter to provide the line information for the error.

macro resetToDecl*(val: typed): untyped = 
  if val.kind == nnkSym:
   # We will implement code here later
  else:
   error("This macro only works with variables." val)
Enter fullscreen mode Exit fullscreen mode

Now though we need to filter out all non-var symbols, since symbols can be any type, proc, variable, ... anything you can know by name. So we'll use the symKind of the symbol to filter out everything but var.

macro resetToDecl*(val: typed): untyped = 
  if val.kind == nnkSym and val.symKind == nskVar:
    ## We'll continue here in next block
  else:
    error("This macro only works on variables", val)
Enter fullscreen mode Exit fullscreen mode

We have done it, we just have variables, but now the question is "How do we get the declaration statement?!", through the power of introspection magic! Nim's macros module exposes multiple other avenues of introspection other than symKind it also has getImpl, getTypeImpl, getType, and so much more. The procedure we will use is of course getImpl cause we want to get the implementation of the symbol(Yes having a symbol is required, batteries not included otherwise).

We can quickly just test to see what the AST is like simply with echo val.getImpl, what we see is a lovely declaration.

IdentDefs
  Sym "a"
  Empty
  IntLit 100
Enter fullscreen mode Exit fullscreen mode

If you squint closely you can see this is quit a predictable situation, all we have to do is assign the val to the val.getImpl[^1] so let us do that!

macro resetToDecl*(val: typed): untyped = 
  if val.kind = nnkSym and val.symKind == nskVar:
    result = nnkAsgn.newTree(val, val.getImpl[^1])
  else:
    error("This macro only works with variables", val) 
Enter fullscreen mode Exit fullscreen mode

I already hear it "Hey is that it, that was easy, there has to be a catch?" and you're right, simply doing var a: int has caused use to get an error that says Error: illformed AST:. The reason for this error is that the macro does not check if the last entry in the symbol is nnkEmpty so all that needs to be done is to make it so if the last entry is empty we emit a default(typeof(val)).

Let us do that:

import std/macros

macro resetToDecl*(val: typed): untyped = 
   if val.kind == nnkSym and val.symKind == nskVar:
     let impl = val.getImpl
     result = nnkAsgn.newTree(val):
       if impl[^1].kind == nnkEmpty:
         newCall("default", newCall("typeOf", val))
       else:
         impl[^1]
   else:
     error("This macro only works on variables", val)
Enter fullscreen mode Exit fullscreen mode

Now to test we've done it properly:

var a = 100
a *= 3
assert a == 300
resetToDecl(a)
assert a == 100


var b: int
b = 300
assert b == 300
b *= 3
assert b == 900
b.resetToDecl
assert b == 0
Enter fullscreen mode Exit fullscreen mode

If you want to see examples I have a variety of packages with mixed complexity of macros:

Slicerator
Oopsie
Constructor
Nimscripter
Nettyrpc
Kashae

💖 💪 🙅 🚩
beef331
Jason Beetham

Posted on September 4, 2020

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Demystification of Macros in Nim
tutorial Demystification of Macros in Nim

September 4, 2020