Source Edit

Low level binary format used by the compiler to store and load various AST and related data.

NB: this is incredibly low level and if you're interested in how the compiler works and less a storage format, you're probably looking for the ic or packed_ast modules to understand the logical format.

Overview

RodFile represents a Rod File (versioned binary format), and the associated data for common interactions such as IO and error tracking (RodFileError). The file format broken up into sections (RodSection) and preceded by a header (see: cookie). The precise layout, section ordering and data following the section are determined by the user. See ic.loadRodFile.

A basic but "wrong" example of the lifecycle:

create or open - create a new one or open an existing
storeHeader - header info
storePrim or storeSeq - save your stuff
close - and we're done

Now read the bits below to understand what's missing.

Issues with the Example

Missing Sections: This is a low level API, so headers and sections need to be stored and loaded by the user, see storeHeader & loadHeader and storeSection & loadSection, respectively.

No Error Handling: The API is centered around IO and prone to error, each operation checks or sets the RodFile.err field. A user of this API needs to handle these appropriately.

API Notes

Valid inputs for Rod files

ASTs, hopes, dreams, and anything as long as it and any children it may have support copyMem. This means anything that is not a pointer and that does not contain a pointer. At a glance these are:

string
objects & tuples (fields are recursed)
sequences AKA seq[T]

Note on error handling style

A flag based approach is used where operations no-op in case of a preexisting error and set the flag if they encounter one.

Misc

'Prim' is short for 'primitive', as in a non-sequence type

Types

RodFile = object f*: File currentSection*: RodSection err*: RodFileError: Source Edit
RodFileError = enum ok, tooBig, cannotOpen, ioFailure, wrongHeader, wrongSection, configMismatch, includeFileChanged: Source Edit
RodSection = enum versionSection, configSection, stringsSection, checkSumsSection, depsSection, numbersSection, exportsSection, hiddenSection, reexportsSection, compilerProcsSection, trmacrosSection, convertersSection, methodsSection, pureEnumsSection, toReplaySection, topLevelSection, bodiesSection, symsSection, typesSection, typeInstCacheSection, procInstCacheSection, attachedOpsSection, methodsPerGenericTypeSection, enumToStringProcsSection, methodsPerTypeSection, dispatchersSection, typeInfoSection, backendFlagsSection, aliveSymsSection, sideChannelSection, namespaceSection, symnamesSection: Source Edit

Procs

proc close(f: var RodFile) {....raises: [], tags: [], forbids: [].}: Source Edit
proc create(filename: string): RodFile {....raises: [], tags: [], forbids: [].}: create the file and open it for writing Source Edit
proc loadHeader(f: var RodFile; cookie = defaultCookie) {....raises: [IOError], tags: [ReadIOEffect], forbids: [].}: Loads the header which is described by cookie. Source Edit
proc loadOrderedTable[K, T](f: var RodFile; s: var OrderedTable[K, T]): T must be compatible with copyMem, see loadPrim Source Edit
proc loadPrim(f: var RodFile; s: var string) {....raises: [IOError], tags: [ReadIOEffect], forbids: [].}: Read a string, the length was stored as a prefix Source Edit
proc loadPrim[T](f: var RodFile; x: var T): Load a non-sequence/string T. Source Edit
proc loadSection(f: var RodFile; expected: RodSection) {....raises: [IOError], tags: [ReadIOEffect], forbids: [].}: read the bytes value of s, sets and error if the section is incorrect. Source Edit
proc loadSeq[T](f: var RodFile; s: var seq[T]): T must be compatible with copyMem, see loadPrim Source Edit
proc open(filename: string): RodFile {....raises: [], tags: [], forbids: [].}: open the file for reading Source Edit
proc storeHeader(f: var RodFile; cookie = defaultCookie) {....raises: [IOError], tags: [WriteIOEffect], forbids: [].}: stores the header which is described by cookie. Source Edit
proc storeOrderedTable[K, T](f: var RodFile; s: OrderedTable[K, T]): Source Edit
proc storePrim(f: var RodFile; s: string) {....raises: [IOError], tags: [WriteIOEffect], forbids: [].}: Stores a string. The len is prefixed to allow for later retreival. Source Edit
proc storePrim[T](f: var RodFile; x: T): Stores a non-sequence/string T. If T doesn't support copyMem and is an object or tuple then the fields are written -- the user from context will need to know which T to load. Source Edit
proc storeSection(f: var RodFile; s: RodSection) {....raises: [IOError], tags: [WriteIOEffect], forbids: [].}: update currentSection and writes the bytes value of s. Source Edit
proc storeSeq[T](f: var RodFile; s: seq[T]): Stores a sequence of Ts, with the len as a prefix for later retrieval. Source Edit

ic/rodfiles