// RLANG — ENGINEERING REPORT

rlang v0.1.11 · R on fusevm · lex/parse → AST → bytecode → Cranelift JIT · MIT · in active development

Docs Reference GitHub
// Color scheme

>_ENGINEERING REPORT

rlang is a compiled R runtime in Rust, hosted on the fusevm bytecode VM and its three-tier Cranelift JIT. This report describes the architecture, the value model, how behaviour is verified, what is not implemented, and the dependency posture. The statements below are facts about the design, the manifest, and the tree — not aspirational metrics.

R
fusevm language host
3
Cranelift JIT tiers
GNU R
differential parity corpus
v0.1.11
version
MIT
license · free / OSS

Summary

GNU R evaluates R by walking a parse tree in C. rlang takes a different path: it lexes and parses R to an AST, lowers that to fusevm bytecode, and runs it on a compiled VM with a Cranelift JIT. Control flow lowers to native VM jumps over native integer counters so the JIT can trace hot loops; R-specific behaviour is served by a thread-local RHost. It joins fusevm and reuses that shared bytecode VM and JIT rather than shipping its own. The binary is Rscript.

The current tree implements the lexer/parser, AST-to-bytecode lowering, the vector heap with attributes (logical, integer, double, character, list, each with NA), recycling and type promotion, copy-on-modify assignment including nested and replacement-function targets, all four index forms plus [[/$/m[i, j], functions with defaults / ... / R's argument matching, lexical closures and <<-, the full ?Syntax operator ladder with user-defined %op% and the native pipe, S3 UseMethod/NextMethod dispatch, the condition system including restarts (tryCatch/try/on.exit/local/withCallingHandlers/withRestarts), its primitives, R's printing rules, the standalone Rscript binary and REPL, the rkyv bytecode cache, the --aot native .fvm executable emitter, the inline-Rust FFI bridge (.rust / .Call), a wasm32-unknown-unknown build, an AOP intercept registry, and an LSP server. Behaviour is diffed against the reference Rscript by a differential parity corpus and 8 runnable examples; both are byte-identical to GNU R 4.6.1. Arguments and defaults are promises and S3 group generics dispatch; data frames, complex numbers, and eigen are not native — see Known gaps.


Hosting on fusevm

rlang contains no virtual machine or JIT of its own. The execution path is:

R source  →  lexer  →  parser → AST  →  lower to fusevm bytecode  →  fusevm VM + Cranelift JIT
                                                 │
                                RHost heap (vectors, attributes, environments, closures)

Shared engine

On native targets fusevm 0.26.7 is pulled from crates.io with the jit, jit-disk-cache, aot, and ffi features; the wasm build uses the bare interpreter. JIT and VM improvements land once and benefit zshrs, stryke, awkrs, elisp, rubylang, and rlang together.

Native control flow

if, for, while, repeat, && and || emit real Jump / JumpIfFalse / JumpIfTrue ops over native Int counters (src/compiler.rs), so the tracing JIT sees ordinary loops instead of a builtin call per iteration.

Persistent native code

jit-disk-cache persists compiled machine code across runs, so warm starts skip recompilation. aot backs Rscript --build (bytecode into the on-disk cache) and Rscript --aot, which links the emitted object against the rlang runtime staticlib into a standalone native .fvm executable.

Inline-Rust FFI

.rust(code) compiles a self-contained Rust block to a cached cdylib via fusevm's ffi feature; R's own .Call(name, …) verb invokes its exports, marshalling length-1 vectors to i64/f64/string and back (src/ffi.rs).

Runs on wasm

The same crate builds for wasm32-unknown-unknown on the bare fusevm interpreter (Cranelift/ffi/LSP/DAP target-gated off) and exports rlang_eval for a web-worker host, with R output routed through a capture buffer (src/wasm.rs).

Handles, not immediates

Unlike a language with scalars, every R value rides through the VM as Value::Obj(u32) — a handle into the host heap — because in R even a "scalar" is a length-one vector that can carry attributes.


Value model

R has no scalars. 1 is a double vector of length one, and any value may carry an insertion-ordered attribute map (names, dim, class, or anything set with attr()). rlang models this directly: RData is a tagged vector (logical / integer / double / character / list), each element optional so NA exists in every atomic type, and every object lives on the RHost heap. Attributes are an IndexMap because attributes(x) exposes their order.

Arithmetic recycles the shorter operand and propagates NA. Logical operators implement three-valued logic — NA & FALSE is FALSE and NA | TRUE is TRUE, because the answer is decided regardless of the missing value. Type promotion in c() follows R's rank order (logical < integer < double < character).

Assignment is copy-on-modify, not reference mutation: z <- y; z[1] <- 99 rebuilds z's vector and re-binds the name, leaving y untouched. Complex targets reduce to that same rule — f(x) <- v compiles to x <- `f<-`(x, v), so names(x) <- v, dim(x) <- v, class(x) <- v, user-defined `f<-`, and nested targets like l$v[2] <- 9 all unwind through rebuild-then-rebind.

Environments are the one reference-typed thing: frames are Rc<RefCell<EnvData>> chained to their enclosure, which gives R's lexical scoping and is what lets <<- assign in the defining frame rather than the caller's. Truthiness follows R, not C or the shell: a condition must be a single non-NA logical, so conditions normalize through a TRUTHY host op before a native branch.


Dispatch and the primitive library

S3 is the object system that exists here. UseMethod walks the value's class vector, tries each generic.class binding in order, and falls back to generic.default; builtin types get their implicit classes. class(), inherits(), structure(), and unclass() are present. Function-position lookup skips non-function bindings, so a variable named c does not shadow the c() primitive.

builtins::PRIMITIVES registers: the apply family (lapply, sapply, vapply, Map, Reduce, Filter, do.call), string and regex functions (paste, sprintf, substr, strsplit, grepl, grep, sub, gsub, trimws), numeric summaries (sum, mean, median, var, sd, cumsum, cumprod, diff, range), sequence and set functions (seq, rep, rev, sort, order, unique, union, intersect, setdiff), coercions and predicates, matrix helpers (matrix, dim, nrow, ncol, t), and the inline-Rust FFI bridge (.rust, .Call). The same corpus backs the LSP's completion and hover.

Printing is its own body of work, because R's console output is part of its observable behaviour: [n] index prefixes with 80-column wrapping, shared decimal widths across a vector, quoted and left-justified character vectors, named-vector column pairs, [i,]/[,j] matrix layout, and $name / [[n]] list sections. Parity is measured on that output, so the formatter is held to the same standard as the evaluator.


Verification

Correctness is measured differentially against the reference implementation, not against a self-recorded baseline.

Live diff against GNU R

cargo run --bin parity runs the corpus (tests/data/parity_corpus.R) through both rlang and the system Rscript and diffs stdout. It needs R on PATH, so CI never runs it.

Frozen replay in CI

tests/parity.rs replays the frozen reference outputs (tests/data/parity_expected.txt) with no R installed, so the corpus is enforced on every build.

Examples are tests

The 8 programs in examples/ embed stopifnot assertions that abort on divergence, and tests/examples.rs runs each through the binary asserting a clean exit and stdout matching the frozen output.

Nothing faked as working

An unimplemented primitive raises could not find function rather than returning a plausible value, and Recall() / Negate() are explicit raising stubs rather than silent approximations.

The corpus and all 8 examples are byte-identical to the reference Rscript from GNU R 4.6.1.


Component status

ComponentStateNotes
lexer / parser → ASTImplementedFull ?Syntax precedence ladder, %op%, native pipe |> (src/lexer.rs, src/parser.rs).
AST → fusevm bytecodeImplementedNative jumps and integer loop counters; no local VM (src/compiler.rs).
vector heap & attributesImplementedFive vector types with NA, recycling, promotion, ordered attribute map (src/host.rs).
indexingImplementedPositive / negative / logical / character, plus [[, $, 2-D m[i, j], and the m[cbind(i, j)] matrix subscript (read and assign, any rank).
assignment & replacement functionsImplemented<-, =, ->, <<-, growing assignment, nested targets, user-defined `f<-`.
functions, closures, argument matchingImplementedArguments and defaults as promises forced at first use, ... forwarding, exact/partial/positional matching.
control flowImplementedif as an expression, for/while/repeat, break/next, short-circuit &&/||.
S3 dispatchImplementedUseMethod/NextMethod over the class vector, implicit classes, .default fallback, and the Ops/Math/Summary group generics.
primitive libraryImplementedApply family, string/regex, numeric summaries, sequence/set, matrix helpers (src/builtins.rs).
R printing rulesImplementedIndex prefixes, 80-column wrapping, shared decimal widths, named-vector, matrix and list layout.
parity harness vs reference RscriptImplementedThe snippet corpus + 8 examples, frozen and replayed in CI (src/bin/parity.rs, tests/parity.rs, tests/examples.rs).
standalone Rscript binary + REPLImplementedFiles, -e one-liners, persistent-host REPL (src/main.rs, src/repl.rs).
rkyv bytecode cacheImplementedVersioned shard at ~/.rlang/scripts.rkyv (src/cache.rs).
AOT (--build cache, --aot executable) / --disasm / --dump-tokens / --dump-astImplemented--build compiles bytecode into the cache; --aot links a standalone native .fvm executable with closures embedded; token stream, AST, and disassembly can be printed (src/aot.rs, src/aot_runtime.rs, src/cli.rs).
Inline-Rust FFI (.rust / .Call) / wasm32 buildImplementedFFI compiles inline Rust to a cached cdylib and calls it through R's .Call; the crate also builds for wasm32-unknown-unknown exporting rlang_eval (src/ffi.rs, src/wasm.rs).
LSP server (--lsp)ImplementedDiagnostics from the runtime parser, hover and completion from the primitive corpus (src/lsp.rs).
AOP call interceptsPartialGlob-matched before/after/around registry is live and tested; the dispatcher does not consult it yet (src/intercepts.rs).
promises / non-standard evaluationImplementedArguments and defaults are promises; substitute(), quote(), match.call() and eval(expr, envir) run natively. NSE programs (dplyr, data.table) re-run in the embedded GNU R.
condition systemImplementedtryCatch, try, on.exit, the simpleError family and restarts (withRestarts/invokeRestart/computeRestarts, muffleWarning/muffleMessage) work, and a condition carries the call it was raised in.
data frames / complex / eigenNot implementedNo native data.frame (it is delegated to the embedded GNU R), no complex numbers, no eigen. factor, table, apply over margins, %*%, outer, cbind/rbind and solve/det/determinant are implemented.
DAP adapter (--dap)PartialHandshake, launch, and run to completion with stdout forwarded as output events; no stepping or breakpoints (src/dap.rs).

Known gaps

The honest list, mirroring BUGS.md. Nothing here is faked as working: an unimplemented primitive raises could not find function, and parity is measured against the real Rscript rather than a self-recorded baseline.

AreaGap
evaluation modelArguments and defaults are promises, forced at first use in the environment that wrote them, and substitute(), quote(), match.call(), sys.call() and eval(expr, envir) work natively. match.arg() and formals() read each default back as the expression written. Non-standard-evaluation programs (dplyr, data.table) re-run in the embedded GNU R when rlang cannot evaluate them.
conditionstryCatch selects a handler by condition class, finally runs either way, try returns a "try-error" string, and on.exit runs however a frame is left. Restarts work, so withCallingHandlers runs its handler at the signalling point and resumes there unless the handler transfers to a restart. A warning and an error both report the call they were raised in, under R's own context rules — a closure makes one, a primitive does not — and the condition object carries it, so conditionCall and print(cond) show it too. Operator warnings (recycling, integer overflow) omit R's In <call> : prefix.
formulas~ builds its formula in the embedded GNU R, so class(y ~ x) needs R installed.
environmentsnew.env(), environment(), local(), parent.frame(n), sys.function(), eval(expr, envir) and $/[[ on an environment work; topenv() does not exist.
typesNo native data frames, complex numbers, raw vectors or Date/POSIXct: data.frame, complex, as.raw and as.Date reach the embedded GNU R. Factors and table are native.
linear algebra%*%, outer, crossprod, solve, det, determinant, apply over margins and cbind/rbind are native, with solve/det/%*% matching R bit for bit; eigen is not.
printingA primitive prints as function (...) .Primitive("name") rather than with R's formals, and a non-global environment shows an address no two processes share.
syntaxelse may start a new line at top level, which R rejects outside { } — so a program R rejects can run here. pkg::name parses and the qualifier is dropped (rlang has one namespace); ? is lexed and unused.
S3 / S4 / R5UseMethod, NextMethod() and the Ops/Math/Summary group generics dispatch natively, with .Generic bound (not .Class/.Method/.Group). No native S4, Reference Classes, or R6; @ parses and reads an attribute, which is not slot semantics.
runtimeNo garbage collection — the RHost heap only grows within a run. The AOP registry is not yet consulted by the dispatcher. The DAP adapter does not step.

Dependency posture

Dependencies are kept foundational and durable — the goal is a crate that still builds cleanly years from now. Direct dependencies from Cargo.toml:

CrateRole
fusevmLanguage-agnostic bytecode VM + three-tier Cranelift JIT, version 0.26.7 (jit, jit-disk-cache, aot, ffi)
clapCLI argument parsing (derive, cargo)
indexmapOrdered names and attribute maps — attributes(x) exposes their order
rkyv / bincode / serdeZero-copy outer shard + serde inner Chunk blobs for the bytecode cache
dirsHome directory for the ~/.rlang cache shard
lsp-server / lsp-types / serde_jsonLSP transport + protocol types (--lsp server)
reedline / nu-ansi-termInteractive REPL line editor and colors
regexgrepl/grep/sub/gsub — R's default patterns are POSIX ERE
globPattern matching for the AOP call-intercept registry
libcPlatform calls behind the binary and LSP stdio path
tempfile (dev)Scratch files for the cache and example test harnesses

Compatibility & longevity

Versioned caches

The rkyv script-cache shard carries a schema version in the key from the first release, so a source or format change misses cleanly instead of loading stale bytecode.

Reference semantics

Behaviour tracks GNU R — recycling, NA propagation, three-valued logic, %%/%/% taking the sign of the divisor, copy-on-modify — as the compatibility target, measured on console output.

Cross-architecture

macOS aarch64 and Linux x86_64 / aarch64 via the Cranelift JIT; portable bytecode underneath.

Standalone crate

An explicit empty [workspace] keeps rlang buildable on its own, independent of the meta repo. The published crate excludes the docs site, CI config, and integration corpus.


Links