Nimrod logo

A new approach to meta programming

Author: Andreas Rumpf

Talk structure

  1. What is Nimrod?
  2. Implementation aspects
  3. "Hello World"
  4. Meta programming features
  5. Optimizing "Hello World" via term rewriting macros
  6. Hoisting via term rewriting macros
  7. Summary of Nimrod's meta programming features

What is Nimrod?

What is Nimrod?

var input: TaintedString

What is Nimrod?

var input: TaintedString
var a = cast[int](gch.stackBottom)

What is Nimrod?

var input: TaintedString
var a = cast[int](gch.stackBottom)
iterator from1to2(): int = yield 1; yield 2

What is Nimrod?

var input: TaintedString
var a = cast[int](gch.stackBottom)
iterator from1to2(): int = yield 1; yield 2
template `!=`(x, y: expr): expr = not (x == y)

Implementation aspects

Implementation aspects

Aporia Screenshot

Hello world

echo "hello ", "world", 99

Hello world

echo "hello ", "world", 99

is rewritten to:

echo([$"hello ", $"world", $99])

Hello world

echo "hello ", "world", 99

is rewritten to:

echo([$"hello ", $"world", $99])
proc `$`(x: MyObject): string = x.s
var obj = MyObject(s: "xyz")
echo obj  # works

Meta programming features

Nimrod's focus is meta programming; macros are used

1. to avoid code duplication / boilerplate:

01    template htmlTag(tag: expr) {.immediate.} =
02      proc tag(): string = "<" & astToStr(tag) & ">"
03
04    htmlTag(br)
05    htmlTag(html)
06
07    echo br()

Produces:

<br>

Meta programming features

2. for control flow abstraction:

01    template once(body: stmt) =
02      var x {.global.} = false
03      if not x:
04        x = true
05        body
06
07    proc p() =
08      once:
09        echo "first call of p"
10      echo "some call of p"
11
12    p()
13    once:
14      echo "new instantiation"
15    p()

Meta programming features

2. for control flow abstraction:

01    template once(body: stmt) =
02      var x {.global.} = false
03      if not x:
04        x = true
05        body
06
07    proc p() =
08      once:
09        echo "first call of p"
10      echo "some call of p"
11
12    p()
13    once:
14      echo "new instantiation"
15    p()

Produces:

first call of p
some call of p
new instantiation
some call of p

Meta programming features

3. for lazy evaluation:

01    template log(msg: string) =
02      if debug:
03        echo msg
04
05    log("x: " & $x & ", y: " & $y)

Meta programming features

4. to implement DSLs:

01    html mainPage:
02      head:
03        title "now look at this"
04      body:
05        ul:
06          li "Nimrod is quite capable"
07
08    echo mainPage()

Produces:

<html>
  <head><title>now look at this</title></head>
  <body>
    <ul>
      <li>Nimrod is quite capable</li>
    </ul>
  </body>
</html>

Meta programming features

Implementation:

01    template html(name: expr, matter: stmt) {.immediate.} =
02      proc name(): string =
03        result = "<html>"
04        matter
05        result.add("</html>")
06
07    template nestedTag(tag: expr) {.immediate.} =
08      template tag(matter: stmt) {.immediate.} =
09        result.add("<" & astToStr(tag) & ">")
10        matter
11        result.add("</" & astToStr(tag) & ">")
12
13    template simpleTag(tag: expr) {.immediate.} =
14      template tag(matter: expr) {.immediate.} =
15        result.add("<$1>$2</$1>" % [astToStr(tag), matter])
16
17    nestedTag body
18    nestedTag head
19    nestedTag ul
20    simpleTag title
21    simpleTag li

Meta programming features

After macro expansion:

template html(name: expr, matter: stmt) {.immediate.} =
  proc name(): string =
    result = "<html>"
    matter
    result.add("</html>")

template head(matter: stmt) {.immediate.} =
  result.add("<" & astToStr(head) & ">")
  matter
  result.add("</" & astToStr(head) & ">")

...

template title(matter: expr) {.immediate.} =
  result.add("<$1>$2</$1>" % [astToStr(title), matter])

template li(matter: expr) {.immediate.} =
  result.add("<$1>$2</$1>" % [astToStr(li), matter])

Meta programming features

01    html mainPage:
02      head:
03        title "now look at this"
04      body:
05        ul:
06          li "Nimrod is quite capable"
07
08    echo mainPage()

Is translated into:

proc mainPage(): string =
  result = "<html>"
  result.add("<head>")
  result.add("<$1>$2</$1>" % ["title", "now look at this"])
  result.add("</head>")
  result.add("<body>")
  result.add("<ul>")
  result.add("<$1>$2</$1>" % ["li", "Nimrod is quite capable"])
  result.add("</ul>")
  result.add("</body>")
  result.add("</html>")

Meta programming features

Compile time function evaluation optimizes 'mainPage()' into:

"<html><head><title>now look at this</title></head><body>..."

Meta programming features

5. to provide user defined optimizations (via term rewriting macros).

Hello world 2.0 (1)

01    type
02      MyObject = object
03        a, b: int
04        s: string
05    let obj = MyObject(a: 3, b: 4, s: "abc")
06    echo obj  # note: calls $ implicitly

Produces (roughly):

(
  a: 3
  b: 4
  s: "abc"
)

How does it work?

How $ for object works

01    # in system module (simplified):
02    proc `$` [T: object](x: T): string =
03      result = "("
04      for name, value in fieldPairs(x):
05        result.add("$1: $2\n" % [name, $value])
06      result.add(")")

Notes:

How $ for object works

result = "("
result.add("$1: $2\n" % ["a", $obj.a])
result.add("$1: $2\n" % ["b", $obj.b])
result.add("$1: $2\n" % ["s", $obj.s])
result.add(")")

Optimizing Hello World (1)

result = "("
result.add("$1: $2\n" % ["a", $obj.a])
result.add("$1: $2\n" % ["b", $obj.b])
result.add("$1: $2\n" % ["s", $obj.s])
result.add(")")

Desired result:

result = `&`("(a: ", $obj.a, "\nb: ", $obj.b, "\ns: ", $obj.s, "\n)")

Optimizing Hello World (2)

We need partial evaluation for '%'.

%'s implementation (simplified):

01    proc `%`(f: string, a: openArray[string]): string =
02      result = ""
03      var i = 0
04      while i < f.len:
05        if f[i] == '$':
06          case f[i+1]
07          of '1'..'9':
08            var j = 0
09            i += 1
10            while f[i] in {'0'..'9'}:
11              j = j * 10 + ord(f[i]) - ord('0'); i += 1
12            result.add(a[j-1])
13          else:
14            invalidFormatString()
15        else:
16          result.add(f[i]); i += 1

Optimizing Hello World (3)

01    macro optFormat{`%`(f, a)}(f: string{lit}, a: openArray[string]): expr =
02      result = newCall("&")
03      var i = 0
04      while i < f.len:
05        if f[i] == '$':
06          case f[i+1]
07          of '1'..'9':
08            var j = 0
09            i += 1
10            while f[i] in {'0'..'9'}:
11              j = j * 10 + ord(f[i]) - ord('0'); i += 1
12            result.add(a[j-1])
13          else:
14            invalidFormatString()
15        else:
16          result.add(newLit(f[i])); i += 1

Implements this optimization:

"$1: $2\n" % ["s", $obj.s]  --> `&`("s", ':', ' ', $obj.s, '\n')

Optimizing Hello World (4)

result = "("
result.add("$1: $2\n" % ["a", $obj.a])
result.add("$1: $2\n" % ["b", $obj.b])
result.add("$1: $2\n" % ["s", $obj.s])
result.add(")")

After partial evaluation:

result = "("
result.add(`&`("a", ':', ' ', $obj.a, '\n'))
result.add(`&`("b", ':', ' ', $obj.b, '\n'))
result.add(`&`("s", ':', ' ', $obj.s, '\n'))
result.add(")")

Optimizing Hello World (4)

result = "("
result.add(`&`("a", ':', ' ', $obj.a, '\n'))
result.add(`&`("b", ':', ' ', $obj.b, '\n'))
result.add(`&`("s", ':', ' ', $obj.s, '\n'))
result.add(")")

After constant folding:

result = "("
result.add(`&`("a: ", $obj.a, '\n'))
result.add(`&`("b: ", $obj.b, '\n'))
result.add(`&`("s: ", $obj.s, '\n'))
result.add(")")

Optimizing Hello World (4)

After constant folding:

result = "("
result.add(`&`("a: ", $obj.a, '\n'))
result.add(`&`("b: ", $obj.b, '\n'))
result.add(`&`("s: ", $obj.s, '\n'))
result.add(")")

Further optimization via term rewriting templates:

template optAdd1{x = y; x.add(z)}(x, y, z: string) =
  x = y & z

template optAdd2{x.add(y); x.add(z)}(x, y, z: string) =
  x.add(y & z)

Optimizing Hello World (4)

result = "("
result.add(`&`("a: ", $obj.a, '\n'))
result.add(`&`("b: ", $obj.b, '\n'))
result.add(`&`("s: ", $obj.s, '\n'))
result.add(")")

template optAdd1{x = y; x.add(z)}(x, y, z: string) =
  x = y & z

Optimizing Hello World (4)

result = "(" & `&`("a: ", $obj.a, '\n')
result.add(`&`("b: ", $obj.b, '\n'))
result.add(`&`("s: ", $obj.s, '\n'))
result.add(")")

template optAdd1{x = y; x.add(z)}(x, y, z: string) =
  x = y & z

Optimizing Hello World (4)

result = "(" & `&`("a: ", $obj.a, '\n')
result.add(`&`("b: ", $obj.b, '\n'))
result.add(`&`("s: ", $obj.s, '\n'))
result.add(")")

template optAdd1{x = y; x.add(z)}(x, y, z: string) =
  x = y & z

Optimizing Hello World (4)

result = "(" & `&`("a: ", $obj.a, '\n') & `&`("b: ", $obj.b, '\n')
result.add(`&`("s: ", $obj.s, '\n'))
result.add(")")

template optAdd1{x = y; x.add(z)}(x, y, z: string) =
  x = y & z

Optimizing Hello World (4)

result = "(" & `&`("a: ", $obj.a, '\n') & `&`("b: ", $obj.b, '\n')
result.add(`&`("s: ", $obj.s, '\n'))
result.add(")")

template optAdd1{x = y; x.add(z)}(x, y, z: string) =
  x = y & z

Optimizing Hello World (4)

result = "(" & `&`("a: ", $obj.a, '\n') & `&`("b: ", $obj.b, '\n') & `&`("s: ", $obj.s, '\n')
result.add(")")

template optAdd1{x = y; x.add(z)}(x, y, z: string) =
  x = y & z

Optimizing Hello World (4)

result = "(" & `&`("a: ", $obj.a, '\n') & `&`("b: ", $obj.b, '\n') & `&`("s: ", $obj.s, '\n')
result.add(")")

template optAdd1{x = y; x.add(z)}(x, y, z: string) =
  x = y & z

Optimizing Hello World (4)

After applying these rules the code is:

result = "(" & `&`("a: ", $obj.a, '\n')) &
               `&`("b: ", $obj.b, '\n')) &
               `&`("s: ", $obj.s, '\n')) &
         ")"

After constant folding (that the compiler performs for us) it becomes:

result = `&`("(a: ", $obj.a, "\nb: ", $obj.b, "\ns: ", $obj.s, "\n)")

DRY means templates

DRY means templates

01    template formatImpl(handleChar: expr) =
02      var i = 0
03      while i < f.len:
04        if f[i] == '$':
05          case f[i+1]
06          of '1'..'9':
07            var j = 0
08            i += 1
09            while f[i] in {'0'..'9'}:
10              j = j * 10 + ord(f[i]) - ord('0'); i += 1
11            result.add(a[j-1])
12          else:
13            invalidFormatString()
14        else:
15          result.add(handleChar(f[i])); i += 1
16
17    proc `%`(f: string, a: openArray[string]): string =
18      template identity(x: expr): expr = x
19      result = ""; formatImpl(identity)
20
21    macro optFormat{`%`(f, a)}(f: string{lit}, a: openArray[string]): expr =
22      result = newCall("&"); formatImpl(newLit)

Hoisting

01    proc re(x: string): Regex =
02      # wrapper around PCRE for instance
03
04    template optRe{re(x)}(x: string{lit}): Regex =
05      var g {.global.} = re(x)
06      g
07
08    template `=~`(s: string, pattern: Regex): bool =
09      when not definedInScope(matches):
10        var matches {.inject.}: array[maxSubPatterns, string]
11      match(s, pattern, matches)
12
13    for line in lines("input.txt"):
14      if line =~ re"(\w+)=(\w+)":
15        echo "key-value pair; key: ", matches[0], " value: ", matches[1]

Summary of meta programming features

You name it, Nimrod got it (except fexprs ;-):

Summary of meta programming features

01    macro check(ex: expr): stmt =
02      var info = ex.lineInfo
03      var expString = ex.toStrLit
04      result = quote do:
05        if not `ex`:
06          echo `info`, ": Check failed: ", `expString`
07
08    check 1 < 2

Thank you

Thank you for listening. We are always looking for contributors: