>_ENGINEERING REPORT
rlang is a compiled R runtime in Rust, hosted on the fusevm bytecode VM and its three-tier Cranelift JIT. This report describes the architecture, the value model, how behaviour is verified, what is not implemented, and the dependency posture. The statements below are facts about the design, the manifest, and the tree — not aspirational metrics.
Summary
GNU R evaluates R by walking a parse tree in C. rlang takes a different path: it lexes and parses R to an AST, lowers that to fusevm bytecode, and runs it on a compiled VM with a Cranelift JIT. Control flow lowers to native VM jumps over native integer counters so the JIT can trace hot loops; R-specific behaviour is served by a thread-local RHost. It joins fusevm alongside zshrs, stryke, awkrs, elisp, and rubylang, and reuses that shared bytecode VM and JIT rather than shipping its own. The binary is Rscript.
The current tree implements the lexer/parser, AST-to-bytecode lowering, the vector heap with attributes (logical, integer, double, character, list, each with NA), recycling and type promotion, copy-on-modify assignment including nested and replacement-function targets, all four index forms plus [[/$/m[i, j], functions with defaults / ... / R's argument matching, lexical closures and <<-, the full ?Syntax operator ladder with user-defined %op% and the native pipe, S3 UseMethod/NextMethod dispatch, the condition system including restarts (tryCatch/try/on.exit/local/withCallingHandlers/withRestarts), 288 primitives, R's printing rules, the standalone Rscript binary and REPL, the rkyv bytecode cache, the --aot native .fvm executable emitter, the inline-Rust FFI bridge (.rust / .Call), a wasm32-unknown-unknown build, an AOP intercept registry, and an LSP server. Behaviour is diffed against the reference Rscript by a 182-snippet differential parity corpus and 8 runnable examples; both are byte-identical to GNU R 4.6.1. Promises and non-standard evaluation, condition calls, data frames, and part of the linear-algebra surface (solve, det, eigen) are not implemented — see Known gaps.
Hosting on fusevm
rlang contains no virtual machine or JIT of its own. The execution path is:
R source → lexer → parser → AST → lower to fusevm bytecode → fusevm VM + Cranelift JIT
│
RHost heap (vectors, attributes, environments, closures)
Shared engine
On native targets fusevm 0.26.0 is pulled from crates.io with the jit, jit-disk-cache, aot, and ffi features; the wasm build uses the bare interpreter. JIT and VM improvements land once and benefit zshrs, stryke, awkrs, elisp, rubylang, and rlang together.
Native control flow
if, for, while, repeat, && and || emit real Jump / JumpIfFalse / JumpIfTrue ops over native Int counters (src/compiler.rs), so the tracing JIT sees ordinary loops instead of a builtin call per iteration.
Persistent native code
jit-disk-cache persists compiled machine code across runs, so warm starts skip recompilation. aot backs Rscript --build (bytecode into the on-disk cache) and Rscript --aot, which links the emitted object against the rlang runtime staticlib into a standalone native .fvm executable.
Inline-Rust FFI
.rust(code) compiles a self-contained Rust block to a cached cdylib via fusevm's ffi feature; R's own .Call(name, …) verb invokes its exports, marshalling length-1 vectors to i64/f64/string and back (src/ffi.rs).
Runs on wasm
The same crate builds for wasm32-unknown-unknown on the bare fusevm interpreter (Cranelift/ffi/LSP/DAP target-gated off) and exports rlang_eval for a web-worker host, with R output routed through a capture buffer (src/wasm.rs).
Handles, not immediates
Unlike a language with scalars, every R value rides through the VM as Value::Obj(u32) — a handle into the host heap — because in R even a "scalar" is a length-one vector that can carry attributes.
Value model
R has no scalars. 1 is a double vector of length one, and any value may carry an insertion-ordered attribute map (names, dim, class, or anything set with attr()). rlang models this directly: RData is a tagged vector (logical / integer / double / character / list), each element optional so NA exists in every atomic type, and every object lives on the RHost heap. Attributes are an IndexMap because attributes(x) exposes their order.
Arithmetic recycles the shorter operand and propagates NA. Logical operators implement three-valued logic — NA & FALSE is FALSE and NA | TRUE is TRUE, because the answer is decided regardless of the missing value. Type promotion in c() follows R's rank order (logical < integer < double < character).
Assignment is copy-on-modify, not reference mutation: z <- y; z[1] <- 99 rebuilds z's vector and re-binds the name, leaving y untouched. Complex targets reduce to that same rule — f(x) <- v compiles to x <- `f<-`(x, v), so names(x) <- v, dim(x) <- v, class(x) <- v, user-defined `f<-`, and nested targets like l$v[2] <- 9 all unwind through rebuild-then-rebind.
Environments are the one reference-typed thing: frames are Rc<RefCell<EnvData>> chained to their enclosure, which gives R's lexical scoping and is what lets <<- assign in the defining frame rather than the caller's. Truthiness follows R, not C or the shell: a condition must be a single non-NA logical, so conditions normalize through a TRUTHY host op before a native branch.
Dispatch and the primitive library
S3 is the object system that exists here. UseMethod walks the value's class vector, tries each generic.class binding in order, and falls back to generic.default; builtin types get their implicit classes. class(), inherits(), structure(), and unclass() are present. Function-position lookup skips non-function bindings, so a variable named c does not shadow the c() primitive.
builtins::PRIMITIVES registers 133 names: the apply family (lapply, sapply, vapply, Map, Reduce, Filter, do.call), string and regex functions (paste, sprintf, substr, strsplit, grepl, grep, sub, gsub, trimws), numeric summaries (sum, mean, median, var, sd, cumsum, cumprod, diff, range), sequence and set functions (seq, rep, rev, sort, order, unique, union, intersect, setdiff), coercions and predicates, matrix helpers (matrix, dim, nrow, ncol, t), and the inline-Rust FFI bridge (.rust, .Call). The same corpus backs the LSP's completion and hover.
Printing is its own body of work, because R's console output is part of its observable behaviour: [n] index prefixes with 80-column wrapping, shared decimal widths across a vector, quoted and left-justified character vectors, named-vector column pairs, [i,]/[,j] matrix layout, and $name / [[n]] list sections. Parity is measured on that output, so the formatter is held to the same standard as the evaluator.
Verification
Correctness is measured differentially against the reference implementation, not against a self-recorded baseline.
Live diff against GNU R
cargo run --bin parity runs the 182-snippet corpus (tests/data/parity_corpus.R) through both rlang and the system Rscript and diffs stdout. It needs R on PATH, so CI never runs it.
Frozen replay in CI
tests/parity.rs replays the frozen reference outputs (tests/data/parity_expected.txt) with no R installed, so the corpus is enforced on every build.
Examples are tests
The 8 programs in examples/ embed stopifnot assertions that abort on divergence, and tests/examples.rs runs each through the binary asserting a clean exit and stdout matching the frozen output.
Nothing faked as working
An unimplemented primitive raises could not find function rather than returning a plausible value, and Recall() / Negate() are explicit raising stubs rather than silent approximations.
The 182-snippet corpus and all 8 examples are byte-identical to the reference Rscript from GNU R 4.6.1.
Component status
| Component | State | Notes |
|---|---|---|
| lexer / parser → AST | Implemented | Full ?Syntax precedence ladder, %op%, native pipe |> (src/lexer.rs, src/parser.rs). |
| AST → fusevm bytecode | Implemented | Native jumps and integer loop counters; no local VM (src/compiler.rs). |
| vector heap & attributes | Implemented | Five vector types with NA, recycling, promotion, ordered attribute map (src/host.rs). |
| indexing | Implemented | Positive / negative / logical / character, plus [[, $, 2-D m[i, j], and the m[cbind(i, j)] matrix subscript (read and assign, any rank). |
| assignment & replacement functions | Implemented | <-, =, ->, <<-, growing assignment, nested targets, user-defined `f<-`. |
| functions, closures, argument matching | Implemented | Defaults referring to other arguments (compiled as a body prologue), ... forwarding, exact/partial/positional matching. |
| control flow | Implemented | if as an expression, for/while/repeat, break/next, short-circuit &&/||. |
| S3 dispatch | Implemented | UseMethod over the class vector, implicit classes, .default fallback. |
| primitive library (288) | Implemented | Apply family, string/regex, numeric summaries, sequence/set, matrix helpers (src/builtins.rs). |
| R printing rules | Implemented | Index prefixes, 80-column wrapping, shared decimal widths, named-vector, matrix and list layout. |
parity harness vs reference Rscript | Implemented | 182 snippets + 8 examples, frozen and replayed in CI (src/bin/parity.rs, tests/parity.rs, tests/examples.rs). |
standalone Rscript binary + REPL | Implemented | Files, -e one-liners, persistent-host REPL (src/main.rs, src/repl.rs). |
| rkyv bytecode cache | Implemented | Versioned shard at ~/.rlang/scripts.rkyv (src/cache.rs). |
AOT (--build cache, --aot executable) / --disasm / --dump-tokens / --dump-ast | Implemented | --build compiles bytecode into the cache; --aot links a standalone native .fvm executable with closures embedded; token stream, AST, and disassembly can be printed (src/aot.rs, src/aot_runtime.rs, src/cli.rs). |
Inline-Rust FFI (.rust / .Call) / wasm32 build | Implemented | FFI compiles inline Rust to a cached cdylib and calls it through R's .Call; the crate also builds for wasm32-unknown-unknown exporting rlang_eval (src/ffi.rs, src/wasm.rs). |
LSP server (--lsp) | Implemented | Diagnostics from the runtime parser, hover and completion from the primitive corpus (src/lsp.rs). |
| AOP call intercepts | Partial | Glob-matched before/after/around registry is live and tested; the dispatcher does not consult it yet (src/intercepts.rs). |
| promises / non-standard evaluation | Not implemented | Arguments evaluate eagerly at the call site. |
| condition system | Partial | tryCatch, try, on.exit, the simpleError family and restarts (withRestarts/invokeRestart/computeRestarts, muffleWarning/muffleMessage) work; conditions carry no call. |
| data frames / complex / part of linear algebra | Not implemented | No native data.frame (it is delegated to the embedded GNU R), no complex numbers, no solve/det/eigen. factor, table, apply over margins, %*%, outer, and cbind/rbind are implemented. |
DAP adapter (--dap) | Partial | Handshake, launch, and run to completion with stdout forwarded as output events; no stepping or breakpoints (src/dap.rs). |
Known gaps
The honest list, mirroring BUGS.md. Nothing here is faked as working: an unimplemented primitive raises could not find function, and parity is measured against the real Rscript rather than a self-recorded baseline.
| Area | Gap |
|---|---|
| evaluation model | Arguments are evaluated eagerly, not as promises, so substitute(), quote(), match.call(), sys.call(), and every non-standard-evaluation idiom are absent (deparse() exists, but of a value — including a closure's source — never of an unevaluated argument). Defaults still behave lazily — they compile into a body prologue, so a default may refer to another argument. |
| conditions | tryCatch selects a handler by condition class, finally runs either way, try returns a "try-error" string, and on.exit runs however a frame is left. Restarts work, so withCallingHandlers runs its handler at the signalling point and resumes there unless the handler transfers to a restart. A warning and an error both report the call they were raised in, under R's own context rules — a closure makes one, a primitive does not — and the condition object carries it, so conditionCall and print(cond) show it too. |
| formulas | ~ lexes but nothing consumes it. |
| environments | Beyond new.env(), environment(), and $/[[ on an environment: no local(), sys.function(), parent.frame(), or eval(expr, envir). |
| types | No native data frames (and so subset, merge, aggregate, read.csv go through the embedded GNU R), no complex numbers, no raw vectors, no Date/POSIXct. Factors and table are native. |
| arrays | Nothing past 2 dimensions: a length-2 dim prints and indexes as a matrix, longer dim vectors are carried but not honoured. |
| linear algebra | No apply over matrix margins, outer, %*%, solve, crossprod; cbind/rbind are not implemented. |
| numerics | Integer overflow wraps to a double instead of producing NA with a warning, because arithmetic is computed in f64 and narrowed back. |
| printing | Small numbers do not switch to scientific notation (R prints 1e-10, rlang prints the expanded decimal). format() is a thin as.character with no nsmall/width/justify/big.mark. No str(), summary(), or dput(). |
| syntax | else may start a new line at top level, which R rejects outside { } — so a program R rejects can run here. pkg::name parses and the qualifier is dropped (rlang has one namespace); ? is lexed and unused. No library()/require() and no package system, so CRAN-dependent programs are out of scope by construction. |
| S3 / S4 / R5 | UseMethod and NextMethod() both work. No S4, Reference Classes, or R6; @ parses and reads an attribute, which is not slot semantics. No user-definable group generics (Ops, Math, Summary), so a class of your own cannot overload + through S3; the factor ones are built in. |
| runtime | No garbage collection — the RHost heap only grows within a run. Closure bodies are cloned per call, which costs on deeply recursive workloads. The AOP registry is not yet consulted by the dispatcher. The DAP adapter does not step. Recall() and Negate() are stubs that raise. |
Dependency posture
Dependencies are kept foundational and durable — the goal is a crate that still builds cleanly years from now. Direct dependencies from Cargo.toml:
| Crate | Role |
|---|---|
| fusevm | Language-agnostic bytecode VM + three-tier Cranelift JIT, version 0.26.0 (jit, jit-disk-cache, aot, ffi) |
| clap | CLI argument parsing (derive, cargo) |
| indexmap | Ordered names and attribute maps — attributes(x) exposes their order |
| rkyv / bincode / serde | Zero-copy outer shard + serde inner Chunk blobs for the bytecode cache |
| dirs | Home directory for the ~/.rlang cache shard |
| lsp-server / lsp-types / serde_json | LSP transport + protocol types (--lsp server) |
| reedline / nu-ansi-term | Interactive REPL line editor and colors |
| regex | grepl/grep/sub/gsub — R's default patterns are POSIX ERE |
| glob | Pattern matching for the AOP call-intercept registry |
| libc | Platform calls behind the binary and LSP stdio path |
| tempfile (dev) | Scratch files for the cache and example test harnesses |
Compatibility & longevity
Versioned caches
The rkyv script-cache shard carries a schema version in the key from the first release, so a source or format change misses cleanly instead of loading stale bytecode.
Reference semantics
Behaviour tracks GNU R — recycling, NA propagation, three-valued logic, %%/%/% taking the sign of the divisor, copy-on-modify — as the compatibility target, measured on console output.
Cross-architecture
macOS aarch64 and Linux x86_64 / aarch64 via the Cranelift JIT; portable bytecode underneath.
Standalone crate
An explicit empty [workspace] keeps rlang buildable on its own, independent of the meta repo. The published crate excludes the docs site, CI config, and integration corpus.
Links
- Docs — index.html
- Primitive reference — reference.html
- Known gaps — BUGS.md
- Source — github.com/MenkeTechnologies/rlang
- Issues — github.com/MenkeTechnologies/rlang/issues
- fusevm — github.com/MenkeTechnologies/fusevm (the shared VM)
- License — MIT (LICENSE)