This module contains the interface to the compiler's abstract syntax tree (AST). Macros operate on this tree.

See also:

• macros tutorial
• macros section in Nim manual

The AST in Nim

This section describes how the AST is modelled with Nim's type system. The AST consists of nodes (NimNode) with a variable number of children. Each node has a field named kind which describes what the node contains:

type
  NimNodeKind = enum     ## kind of a node; only explanatory
    nnkNone,             ## invalid node kind
    nnkEmpty,            ## empty node
    nnkIdent,            ## node contains an identifier
    nnkIntLit,           ## node contains an int literal (example: 10)
    nnkStrLit,           ## node contains a string literal (example: "abc")
    nnkNilLit,           ## node contains a nil literal (example: nil)
    nnkCaseStmt,         ## node represents a case statement
    ...                  ## many more
  
  NimNode = ref NimNodeObj
  NimNodeObj = object
    case kind: NimNodeKind           ## the node's kind
    of nnkNone, nnkEmpty, nnkNilLit:
      discard                        ## node contains no additional fields
    of nnkCharLit..nnkUInt64Lit:
      intVal: BiggestInt             ## the int literal
    of nnkFloatLit..nnkFloat64Lit:
      floatVal: BiggestFloat         ## the float literal
    of nnkStrLit..nnkTripleStrLit, nnkCommentStmt, nnkIdent, nnkSym:
      strVal: string                 ## the string literal
    else:
      sons: seq[NimNode]             ## the node's sons (or children)

For the NimNode type, the [] operator has been overloaded: n[i] is n's i-th child.

To specify the AST for the different Nim constructs, the notation nodekind(son1, son2, ...) or nodekind(value) or nodekind(field=value) is used.

Some child may be missing. A missing child is a node of kind nnkEmpty; a child can never be nil.

Leaf nodes/Atoms

A leaf of the AST often corresponds to a terminal symbol in the concrete syntax. Note that the default float in Nim maps to float64 such that the default AST for a float is nnkFloat64Lit as below.

Nim expression	Corresponding AST
42	nnkIntLit(intVal = 42)
42'i8	nnkInt8Lit(intVal = 42)
42'i16	nnkInt16Lit(intVal = 42)
42'i32	nnkInt32Lit(intVal = 42)
42'i64	nnkInt64Lit(intVal = 42)
42'u8	nnkUInt8Lit(intVal = 42)
42'u16	nnkUInt16Lit(intVal = 42)
42'u32	nnkUInt32Lit(intVal = 42)
42'u64	nnkUInt64Lit(intVal = 42)
42.0	nnkFloat64Lit(floatVal = 42.0)
42.0'f32	nnkFloat32Lit(floatVal = 42.0)
42.0'f64	nnkFloat64Lit(floatVal = 42.0)
"abc"	nnkStrLit(strVal = "abc")
r"abc"	nnkRStrLit(strVal = "abc")
"""abc"""	nnkTripleStrLit(strVal = "abc")
' '	nnkCharLit(intVal = 32)
nil	nnkNilLit()
myIdentifier	nnkIdent(strVal = "myIdentifier")
myIdentifier	after lookup pass: nnkSym(strVal = "myIdentifier", ...)

Identifiers are nnkIdent nodes. After the name lookup pass these nodes get transferred into nnkSym nodes.

Calls/expressions

Command call

Concrete syntax:

echo "abc", "xyz"

AST:

nnkCommand(
  nnkIdent("echo"),
  nnkStrLit("abc"),
  nnkStrLit("xyz")
)

Call with ()

Concrete syntax:

echo("abc", "xyz")

AST:

nnkCall(
  nnkIdent("echo"),
  nnkStrLit("abc"),
  nnkStrLit("xyz")
)

Infix operator call

Concrete syntax:

"abc" & "xyz"

AST:

nnkInfix(
  nnkIdent("&"),
  nnkStrLit("abc"),
  nnkStrLit("xyz")
)

Note that with multiple infix operators, the command is parsed by operator precedence.

Concrete syntax:

5 + 3 * 4

AST:

nnkInfix(
  nnkIdent("+"),
  nnkIntLit(5),
  nnkInfix(
    nnkIdent("*"),
    nnkIntLit(3),
    nnkIntLit(4)
  )
)

As a side note, if you choose to use infix operators in a prefix form, the AST behaves as a parenthetical function call with nnkAccQuoted, as follows:

Concrete syntax:

`+`(3, 4)

AST:

nnkCall(
  nnkAccQuoted(
    nnkIdent("+")
  ),
  nnkIntLit(3),
  nnkIntLit(4)
)

Prefix operator call

Concrete syntax:

? "xyz"

AST:

nnkPrefix(
  nnkIdent("?"),
  nnkStrLit("abc")
)

Postfix operator call

Note: There are no postfix operators in Nim. However, the nnkPostfix node is used for the asterisk export marker *:

Concrete syntax:

identifier*

AST:

nnkPostfix(
  nnkIdent("*"),
  nnkIdent("identifier")
)

Call with named arguments

Concrete syntax:

writeLine(file=stdout, "hallo")

AST:

nnkCall(
  nnkIdent("writeLine"),
  nnkExprEqExpr(
    nnkIdent("file"),
    nnkIdent("stdout")
  ),
  nnkStrLit("hallo")
)

Call with raw string literal

This is used, for example, in the bindSym examples here and with re"some regexp" in the regular expression module.

Concrete syntax:

echo"abc"

AST:

nnkCallStrLit(
  nnkIdent("echo"),
  nnkRStrLit("hello")
)

Dereference operator []

Concrete syntax:

x[]

AST:

nnkDerefExpr(nnkIdent("x"))

Addr operator

Concrete syntax:

addr(x)

AST:

nnkAddr(nnkIdent("x"))

Cast operator

Concrete syntax:

cast[T](x)

AST:

nnkCast(nnkIdent("T"), nnkIdent("x"))

Object access operator .

Concrete syntax:

x.y

AST:

nnkDotExpr(nnkIdent("x"), nnkIdent("y"))

If you use Nim's flexible calling syntax (as in x.len()), the result is the same as above but wrapped in an nnkCall.

Array access operator []

Concrete syntax:

x[y]

AST:

nnkBracketExpr(nnkIdent("x"), nnkIdent("y"))

Parentheses

Parentheses for affecting operator precedence or tuple construction are built with the nnkPar node.

Concrete syntax:

(1, 2, (3))

AST:

nnkPar(nnkIntLit(1), nnkIntLit(2), nnkPar(nnkIntLit(3)))

Curly braces

Curly braces are used as the set constructor.

Concrete syntax:

{1, 2, 3}

AST:

nnkCurly(nnkIntLit(1), nnkIntLit(2), nnkIntLit(3))

When used as a table constructor, the syntax is different.

Concrete syntax:

{a: 3, b: 5}

AST:

nnkTableConstr(
  nnkExprColonExpr(nnkIdent("a"), nnkIntLit(3)),
  nnkExprColonExpr(nnkIdent("b"), nnkIntLit(5))
)

Brackets

Brackets are used as the array constructor.

Concrete syntax:

[1, 2, 3]

AST:

nnkBracket(nnkIntLit(1), nnkIntLit(2), nnkIntLit(3))

Ranges

Ranges occur in set constructors, case statement branches, or array slices. Internally, the node kind nnkRange is used, but when constructing the AST, construction with .. as an infix operator should be used instead.

Concrete syntax:

1..3

AST:

nnkInfix(
  nnkIdent(".."),
  nnkIntLit(1),
  nnkIntLit(3)
)

Example code:

macro genRepeatEcho(): stmt =
  result = newNimNode(nnkStmtList)
  
  var forStmt = newNimNode(nnkForStmt) # generate a for statement
  forStmt.add(ident("i")) # use the variable `i` for iteration
  
  var rangeDef = newNimNode(nnkInfix).add(
    ident("..")).add(
    newIntLitNode(3),newIntLitNode(5)) # iterate over the range 3..5
  
  forStmt.add(rangeDef)
  forStmt.add(newCall(ident("echo"), newIntLitNode(3))) # meat of the loop
  result.add(forStmt)

genRepeatEcho() # gives:
                # 3
                # 3
                # 3

If expression

The representation of the if expression is subtle, but easy to traverse.

Concrete syntax:

if cond1: expr1 elif cond2: expr2 else: expr3

AST:

nnkIfExpr(
  nnkElifExpr(cond1, expr1),
  nnkElifExpr(cond2, expr2),
  nnkElseExpr(expr3)
)

Documentation Comments

Double-hash (##) comments in the code actually have their own format, using strVal to get and set the comment text. Single-hash (#) comments are ignored.

Concrete syntax:

## This is a comment
## This is part of the first comment
stmt1
## Yet another

AST:

nnkCommentStmt() # only appears once for the first two lines!
stmt1
nnkCommentStmt() # another nnkCommentStmt because there is another comment
                 # (separate from the first)

Pragmas

One of Nim's cool features is pragmas, which allow fine-tuning of various aspects of the language. They come in all types, such as adorning procs and objects, but the standalone emit pragma shows the basics with the AST.

Concrete syntax:

{.emit: "#include <stdio.h>".}

AST:

nnkPragma(
  nnkExprColonExpr(
    nnkIdent("emit"),
    nnkStrLit("#include <stdio.h>") # the "argument"
  )
)

As many nnkIdent appear as there are pragmas between {..}. Note that the declaration of new pragmas is essentially the same:

Concrete syntax:

{.pragma: cdeclRename, cdecl.}

AST:

nnkPragma(
  nnkExprColonExpr(
    nnkIdent("pragma"), # this is always first when declaring a new pragma
    nnkIdent("cdeclRename") # the name of the pragma
  ),
  nnkIdent("cdecl")
)

Statements

If statement

The representation of the if statement is subtle, but easy to traverse. If there is no else branch, no nnkElse child exists.

Concrete syntax:

if cond1:
  stmt1
elif cond2:
  stmt2
elif cond3:
  stmt3
else:
  stmt4

AST:

nnkIfStmt(
  nnkElifBranch(cond1, stmt1),
  nnkElifBranch(cond2, stmt2),
  nnkElifBranch(cond3, stmt3),
  nnkElse(stmt4)
)

When statement

Like the if statement, but the root has the kind nnkWhenStmt.

Assignment

Concrete syntax:

x = 42

AST:

nnkAsgn(nnkIdent("x"), nnkIntLit(42))

This is not the syntax for assignment when combined with var, let, or const.

Statement list

Concrete syntax:

stmt1
stmt2
stmt3

AST:

nnkStmtList(stmt1, stmt2, stmt3)

Case statement

Concrete syntax:

case expr1
of expr2, expr3..expr4:
  stmt1
of expr5:
  stmt2
elif cond1:
  stmt3
else:
  stmt4

AST:

nnkCaseStmt(
  expr1,
  nnkOfBranch(expr2, nnkRange(expr3, expr4), stmt1),
  nnkOfBranch(expr5, stmt2),
  nnkElifBranch(cond1, stmt3),
  nnkElse(stmt4)
)

The nnkElifBranch and nnkElse parts may be missing.

While statement

Concrete syntax:

while expr1:
  stmt1

AST:

nnkWhileStmt(expr1, stmt1)

For statement

Concrete syntax:

for ident1, ident2 in expr1:
  stmt1

AST:

nnkForStmt(ident1, ident2, expr1, stmt1)

Try statement

Concrete syntax:

try:
  stmt1
except e1, e2:
  stmt2
except e3:
  stmt3
except:
  stmt4
finally:
  stmt5

AST:

nnkTryStmt(
  stmt1,
  nnkExceptBranch(e1, e2, stmt2),
  nnkExceptBranch(e3, stmt3),
  nnkExceptBranch(stmt4),
  nnkFinally(stmt5)
)

Return statement

Concrete syntax:

return expr1

AST:

nnkReturnStmt(expr1)

Yield statement

Like return, but with nnkYieldStmt kind.

nnkYieldStmt(expr1)

Discard statement

Like return, but with nnkDiscardStmt kind.

nnkDiscardStmt(expr1)

Continue statement

Concrete syntax:

continue

AST:

nnkContinueStmt()

Break statement

Concrete syntax:

break otherLocation

AST:

nnkBreakStmt(nnkIdent("otherLocation"))

If break is used without a jump-to location, nnkEmpty replaces nnkIdent.

Block statement

Concrete syntax:

block name:

AST:

nnkBlockStmt(nnkIdent("name"), nnkStmtList(...))

A block doesn't need an name, in which case nnkEmpty is used.

Asm statement

Concrete syntax:

asm """
  some asm
"""

AST:

nnkAsmStmt(
  nnkEmpty(), # for pragmas
  nnkTripleStrLit("some asm"),
)

Import section

Nim's import statement actually takes different variations depending on what keywords are present. Let's start with the simplest form.

Concrete syntax:

import math

AST:

nnkImportStmt(nnkIdent("math"))

With except, we get nnkImportExceptStmt.

Concrete syntax:

import math except pow

AST:

nnkImportExceptStmt(nnkIdent("math"),nnkIdent("pow"))

Note that import math as m does not use a different node; rather, we use nnkImportStmt with as as an infix operator.

Concrete syntax:

import strutils as su

AST:

nnkImportStmt(
  nnkInfix(
    nnkIdent("as"),
    nnkIdent("strutils"),
    nnkIdent("su")
  )
)

From statement

If we use from ... import, the result is different, too.

Concrete syntax:

from math import pow

AST:

nnkFromStmt(nnkIdent("math"), nnkIdent("pow"))

Using from math as m import pow works identically to the as modifier with the import statement, but wrapped in nnkFromStmt.

Export statement

When you are making an imported module accessible by modules that import yours, the export syntax is pretty straightforward.

Concrete syntax:

export unsigned

AST:

nnkExportStmt(nnkIdent("unsigned"))

Similar to the import statement, the AST is different for export ... except.

Concrete syntax:

export math except pow # we're going to implement our own exponentiation

AST:

nnkExportExceptStmt(nnkIdent("math"),nnkIdent("pow"))

Include statement

Like a plain import statement but with nnkIncludeStmt.

Concrete syntax:

include blocks

AST:

nnkIncludeStmt(nnkIdent("blocks"))

Var section

Concrete syntax:

var a = 3

AST:

nnkVarSection(
  nnkIdentDefs(
    nnkIdent("a"),
    nnkEmpty(), # or nnkIdent(...) if the variable declares the type
    nnkIntLit(3),
  )
)

Note that either the second or third (or both) parameters above must exist, as the compiler needs to know the type somehow (which it can infer from the given assignment).

This is not the same AST for all uses of var. See Procedure declaration for details.

Let section

This is equivalent to var, but with nnkLetSection rather than nnkVarSection.

Concrete syntax:

let a = 3

AST:

nnkLetSection(
  nnkIdentDefs(
    nnkIdent("a"),
    nnkEmpty(), # or nnkIdent(...) for the type
    nnkIntLit(3),
  )
)

Const section

Concrete syntax:

const a = 3

AST:

nnkConstSection(
  nnkConstDef( # not nnkConstDefs!
    nnkIdent("a"),
    nnkEmpty(), # or nnkIdent(...) if the variable declares the type
    nnkIntLit(3), # required in a const declaration!
  )
)

Type section

Starting with the simplest case, a type section appears much like var and const.

Concrete syntax:

type A = int

AST:

nnkTypeSection(
  nnkTypeDef(
    nnkIdent("A"),
    nnkEmpty(),
    nnkIdent("int")
  )
)

Declaring distinct types is similar, with the last nnkIdent wrapped in nnkDistinctTy.

Concrete syntax:

type MyInt = distinct int

AST:

# ...
nnkTypeDef(
  nnkIdent("MyInt"),
  nnkEmpty(),
  nnkDistinctTy(
    nnkIdent("int")
  )
)

If a type section uses generic parameters, they are treated here:

Concrete syntax:

type A[T] = expr1

AST:

nnkTypeSection(
  nnkTypeDef(
    nnkIdent("A"),
    nnkGenericParams(
      nnkIdentDefs(
        nnkIdent("T"),
        nnkEmpty(), # if the type is declared with options, like
                    # ``[T: SomeInteger]``, they are given here
        nnkEmpty(),
      )
    )
    expr1,
  )
)

Note that not all nnkTypeDef utilize nnkIdent as their parameter. One of the most common uses of type declarations is to work with objects.

Concrete syntax:

type IO = object of RootObj

AST:

# ...
nnkTypeDef(
  nnkIdent("IO"),
  nnkEmpty(),
  nnkObjectTy(
    nnkEmpty(), # no pragmas here
    nnkOfInherit(
      nnkIdent("RootObj") # inherits from RootObj
    )
    nnkEmpty()
  )
)

Nim's object syntax is rich. Let's take a look at an involved example in its entirety to see some of the complexities.

Concrete syntax:

type Obj[T] = object {.inheritable.}
  name: string
  case isFat: bool
  of true:
    m: array[100_000, T]
  of false:
    m: array[10, T]

AST:

# ...
nnkObjectTy(
  nnkPragma(
    nnkIdent("inheritable")
  ),
  nnkEmpty(),
  nnkRecList( # list of object parameters
    nnkIdentDefs(
      nnkIdent("name"),
      nnkIdent("string"),
      nnkEmpty()
    ),
    nnkRecCase( # case statement within object (not nnkCaseStmt)
      nnkIdentDefs(
        nnkIdent("isFat"),
        nnkIdent("bool"),
        nnkEmpty()
      ),
      nnkOfBranch(
        nnkIdent("true"),
        nnkRecList( # again, a list of object parameters
          nnkIdentDefs(
            nnkIdent("m"),
            nnkBracketExpr(
              nnkIdent("array"),
              nnkIntLit(100000),
              nnkIdent("T")
            ),
            nnkEmpty()
        )
      ),
      nnkOfBranch(
        nnkIdent("false"),
        nnkRecList(
          nnkIdentDefs(
            nnkIdent("m"),
            nnkBracketExpr(
              nnkIdent("array"),
              nnkIntLit(10),
              nnkIdent("T")
            ),
            nnkEmpty()
          )
        )
      )
    )
  )
)

Using an enum is similar to using an object.

Concrete syntax:

type X = enum
  First

AST:

# ...
nnkEnumTy(
  nnkEmpty(),
  nnkIdent("First") # you need at least one nnkIdent or the compiler complains
)

The usage of concept (experimental) is similar to objects.

Concrete syntax:

type Con = concept x,y,z
  (x & y & z) is string

AST:

# ...
nnkTypeClassTy( # note this isn't nnkConceptTy!
  nnkArglist(
    # ... idents for x, y, z
  )
  # ...
)

Static types, like static[int], use nnkIdent wrapped in nnkStaticTy.

Concrete syntax:

type A[T: static[int]] = object

AST:

# ... within nnkGenericParams
nnkIdentDefs(
  nnkIdent("T"),
  nnkStaticTy(
    nnkIdent("int")
  ),
  nnkEmpty()
)
# ...

In general, declaring types mirrors this syntax (i.e., nnkStaticTy for static, etc.). Examples follow (exceptions marked by *):

Nim type	Corresponding AST
static	nnkStaticTy
tuple	nnkTupleTy
var	nnkVarTy
ptr	nnkPtrTy
ref	nnkRefTy
distinct	nnkDistinctTy
enum	nnkEnumTy
concept	nnkTypeClassTy*
array	nnkBracketExpr(nnkIdent("array"),...*
proc	nnkProcTy
iterator	nnkIteratorTy
object	nnkObjectTy

Take special care when declaring types as proc. The behavior is similar to Procedure declaration, below, but does not treat nnkGenericParams. Generic parameters are treated in the type, not the proc itself.

Concrete syntax:

type MyProc[T] = proc(x: T)

AST:

# ...
nnkTypeDef(
  nnkIdent("MyProc"),
  nnkGenericParams( # here, not with the proc
    # ...
  )
  nnkProcTy( # behaves like a procedure declaration from here on
    nnkFormalParams(
      # ...
    )
  )
)

The same syntax applies to iterator (with nnkIteratorTy), but does not apply to converter or template.

Mixin statement

Concrete syntax:

mixin x

AST:

nnkMixinStmt(nnkIdent("x"))

Bind statement

Concrete syntax:

bind x

AST:

nnkBindStmt(nnkIdent("x"))

Procedure declaration

Let's take a look at a procedure with a lot of interesting aspects to get a feel for how procedure calls are broken down.

Concrete syntax:

proc hello*[T: SomeInteger](x: int = 3, y: float32): int {.inline.} = discard

AST:

nnkProcDef(
  nnkPostfix(nnkIdent("*"), nnkIdent("hello")), # the exported proc name
  nnkEmpty(), # patterns for term rewriting in templates and macros (not procs)
  nnkGenericParams( # generic type parameters, like with type declaration
    nnkIdentDefs(
      nnkIdent("T"),
      nnkIdent("SomeInteger"),
      nnkEmpty()
    )
  ),
  nnkFormalParams(
    nnkIdent("int"), # the first FormalParam is the return type. nnkEmpty() if there is none
    nnkIdentDefs(
      nnkIdent("x"),
      nnkIdent("int"), # type type (required for procs, not for templates)
      nnkIntLit(3) # a default value
    ),
    nnkIdentDefs(
      nnkIdent("y"),
      nnkIdent("float32"),
      nnkEmpty()
    )
  ),
  nnkPragma(nnkIdent("inline")),
  nnkEmpty(), # reserved slot for future use
  nnkStmtList(nnkDiscardStmt(nnkEmpty())) # the meat of the proc
)

There is another consideration. Nim has flexible type identification for its procs. Even though proc(a: int, b: int) and proc(a, b: int) are equivalent in the code, the AST is a little different for the latter.

Concrete syntax:

proc(a, b: int)

AST:

# ...AST as above...
nnkFormalParams(
  nnkEmpty(), # no return here
  nnkIdentDefs(
    nnkIdent("a"), # the first parameter
    nnkIdent("b"), # directly to the second parameter
    nnkIdent("int"), # their shared type identifier
    nnkEmpty(), # default value would go here
  )
),
# ...

When a procedure uses the special var type return variable, the result is different from that of a var section.

Concrete syntax:

proc hello(): var int

AST:

# ...
nnkFormalParams(
  nnkVarTy(
    nnkIdent("int")
  )
)

Iterator declaration

The syntax for iterators is similar to procs, but with nnkIteratorDef replacing nnkProcDef.

Concrete syntax:

iterator nonsense[T](x: seq[T]): float {.closure.} = ...

AST:

nnkIteratorDef(
  nnkIdent("nonsense"),
  nnkEmpty(),
  ...
)

Converter declaration

A converter is similar to a proc.

Concrete syntax:

converter toBool(x: float): bool

AST:

nnkConverterDef(
  nnkIdent("toBool"),
  # ...
)

Template declaration

Templates (as well as macros, as we'll see) have a slightly expanded AST when compared to procs and iterators. The reason for this is [term-rewriting macros](manual.html#term-rewriting-macros). Notice the nnkEmpty() as the second argument to nnkProcDef and nnkIteratorDef above? That's where the term-rewriting macros go.

Concrete syntax:

template optOpt{expr1}(a: int): int

AST:

nnkTemplateDef(
  nnkIdent("optOpt"),
  nnkStmtList( # instead of nnkEmpty()
    expr1
  ),
  # follows like a proc or iterator
)

If the template does not have types for its parameters, the type identifiers inside nnkFormalParams just becomes nnkEmpty.

Macro declaration

Macros behave like templates, but nnkTemplateDef is replaced with nnkMacroDef.

Special node kinds

There are several node kinds that are used for semantic checking or code generation. These are accessible from this module, but should not be used. Other node kinds are especially designed to make AST manipulations easier. These are explained here.

To be written.