>_ENGINEERING REPORT
tclrs is a Tcl frontend in Rust, hosted on the fusevm bytecode VM. This report describes the architecture, the value model, the two hooks that carry Tcl's semantics, how the tree is tested against the reference interpreter, and the dependency posture. The statements below are facts about the design and the manifest, not aspirational metrics.
Summary
Tcl 9 evaluates through its own bytecode engine and object model; tclrs takes a different path. It parses a script once — resolving every substitution the grammar permits at parse time — and lowers each command to fusevm bytecode, the shared bytecode sixteen other language frontends already emit. There is no interpreter loop and no code generator in this crate: execution and codegen belong to the VM. Arithmetic, comparison, bitwise, shift, and short-circuit logic lower to native fusevm ops; the operators whose Tcl meaning differs from the VM's generic one take frontend extension ops; operands the VM cannot compute on natively take the numeric hook.
The tree carries the parser (all twelve rules of Tcl(n)), the compiler, the runtime driver, and a command surface of set, puts, expr, incr, unset, append, if/elseif/else, while, for, foreach, switch, break, continue, proc, return, global, catch, error, eval, coroutine/yield/yieldto/info coroutine, the thirteen list commands, array, dict, array variables, format, the string ensemble, the info ensemble, regexp/regsub, uplevel and apply — with command substitution of any of them. There is a tclrs binary with a REPL, an ahead-of-time compiler, fusevm's Cranelift tiers armed on every VM, and a language server and debug adapter over stdio. Everything outside the surface is refused with a Tcl-shaped message rather than approximated. See BUGS.md for the honest ledger.
Hosting on fusevm
tclrs contains no virtual machine and no code generator of its own. The execution path is:
Tcl script → parser (Script/Command/Word) → lower to fusevm bytecode → fusevm VM
│
numeric hook (string operands, integer overflow)
extension ops (floored / and %, integral **, normalize)
Shared engine
fusevm is pulled from crates.io. VM work lands once and benefits zshrs, stryke, awkrs, vimlrs, elisprs, rubylang, pythonrs, phplang, node-js, rlang, go-rs, arb, the JVM frontends, and tclrs together.
Compiled once, not per evaluation
Braces suppress substitution, so a braced body — an if or while body, a braced expr expression — is fully known at parse time and lowers to bytecode once. Words carry a braced flag for exactly this decision.
Native arithmetic stays native
+ - *, the comparisons, & ^ | ~ << >>, and short-circuiting && / || lower to native fusevm ops, so the arithmetic the JIT tiers care about stays visible to them.
Tiers armed, and reported
fusevm is depended on with jit, jit-disk-cache and aot, and every VM this crate builds arms the tracing JIT. A hot while or for loop inside a proc reaches a compiled trace: its locals are frame slots, and every loop is emitted rotated into the do-while shape fusevm's trace compiler accepts. The same loop at a script's top level does not — a top-level variable is a VM global, which no tier takes. tclrs --tiers asks fusevm's own predicates rather than assuming, and the README names both halves. Ahead-of-time compilation reaches native code in either spelling, and the benchmarks say by how much.
Value model
Tcl's value model needs no object heap on top of fusevm's. Strings, integers, and floats map onto fusevm::Value directly, so a value produced as a number stays a number in a VM slot and only acquires a string representation when something asks for one — which is where the reference implementation spends time in hot loops. Variables are addressed by name through the VM's slot table; there are no call frames yet, because there is no proc.
Two hooks carry all of the language-specific behavior. The numeric hook catches operands the VM cannot compute on natively: an operand that parses as a number is one — with leading and trailing whitespace tolerated and the 0x / 0o / 0b radix prefixes accepted — comparisons fall back to string order when it does not, and arithmetic on a non-number is an error. That is what makes expr {"10" < "9"} false while expr {"abc" < "abd"} is true. The extension handler implements the operators whose Tcl meaning differs from the VM's generic one: / and % floor toward negative infinity (-57 / 10 is -6, -57 % 10 is 3), ** stays integral for integral operands, and a normalize op converts a VM-native result into its Tcl value — a boolean to 1 or 0, a double to Tcl's double format (the shortest representation that reads back exactly, never looking like an integer, exponential outside the positional range).
Tcl promotes an overflowing integer to arbitrary precision. tclrs has no bignum yet, so an operation that overflows i64 reports integer value too large to represent rather than wrapping silently.
Component status
| Component | State | Notes |
|---|---|---|
| parser — twelve rules | Implemented | Command/word splitting with line tracking, quotes, brace nesting, eager command substitution, the four variable forms, the full backslash table including the backslash-newline pre-pass, first-word comments (src/parser.rs). |
| Script → fusevm bytecode | Implemented | Statically tracked stack depth, backpatched control flow, no local VM (src/compiler.rs). |
expr grammar | Implemented | Its own recursive-descent grammar at expr(n) precedence, compiled straight from a braced word with no runtime parse (src/expr.rs). |
| numeric hook + extension ops | Implemented | String operands, integer overflow, floored / and %, integral **, value normalization, Tcl double formatting (src/runtime.rs). |
| commands | Implemented | set, puts (-nonewline), expr, incr, unset, append, if/elseif/else, while, for, foreach, switch, break, continue, proc, return, global, catch, error, eval, the coroutine commands, the thirteen list commands, array, dict, format, the string ensemble, and command substitution of any of them. |
| loop exits | Implemented | break and continue unwind with a compile-time-known pop count rather than a runtime unwinder, in foreach as well as while. |
| list syntax | Implemented | String ⇄ elements ported from TclFindElement and TclScanElement/TclConvertElement, with Tcl's index grammar and glob matcher (src/list.rs). |
list commands / in / ni | Implemented | Each lowers to one extension op; lsort reproduces the reference merge sort so -unique keeps the element tclsh keeps (src/cmd_list.rs). Unbuilt options are errors, not no-ops. |
{*} expansion | Implemented | One ext::EXPAND_CALL per command that has one: the words are spliced by list rules when the command runs, and the callee is resolved then — a procedure of the interpreter, a command this frontend compiles, or a command Tk registered (src/procs.rs). |
the binary, the REPL, eval | Implemented | A file, -c script, or stdin — a REPL when stdin is a terminal. An Interp holds the variables between evaluations over a source-keyed chunk cache; eval compiles a script built at run time against that state. |
regexp / regsub | Implemented | -nocase, -all, -inline, -indices, -line, -lineanchor, -linestop, -expanded, -start and --, plus switch -regexp and lsearch -regexp. The engine is a finite automaton, so the two constructs that need backtracking — back-references and look-ahead — are refused by name rather than approximated (src/regexp.rs). |
| arbitrary-precision integers | Implemented | Tcl 9's integers are unbounded and so are these: an i64 overflow promotes through the numeric hook rather than raising, and every chunk carries int_overflow_deopt so the JIT and the ahead-of-time compiler deopt into the hook instead of wrapping (src/runtime.rs). |
math functions, channel IO, upvar, namespace | Refused | A math call parses into an Expr::Call the compiler refuses; an unknown command is a compile-time error. |
| JIT / AOT | Linked and measured | The jit, jit-disk-cache and aot features are on; --aot reaches native code, and --tiers reports that the tracing tier does too for every counted loop, whether its counter is a procedure's frame slot or a script's top-level VM global. What a loop still fails on is an extension op in its body — foreach and dict for carry their loop state in one. |
| LSP / DAP | Implemented | --lsp answers diagnostics, completion, hover, signature help and document symbols over stdio, from the compiler's own tables and the cursor module the REPL completes with (src/lsp.rs). --dap gives breakpoints, stepping, stack frame and variables by stopping on markers compiler::compile_debug emits and an ordinary compilation does not (src/dap.rs). Both are tested by speaking the real protocol to the real process. |
inline Rust — rust { ... } | Implemented | Rewritten before parsing into __rust_compile, compiled and dlopened through fusevm::ffi, cached by the SHA-256 of the block's body, and registered as the block is lowered, so its exports are known command names on the next line (src/rust_ffi.rs). |
| dumps / completion / man pages | Implemented | --disasm, --dump-tokens and --dump-ast; the zsh completion completions/_tclrs; tclrs(1) and the all-in-one tclrsall(1); and reference.html, generated from the compiler's tables by cargo run --bin gen-docs. |
Differential testing
tclsh 9.0.4 is the specification, and the suite compares against it directly — no expected output in the repository is written by hand, so a misreading of the grammar or of Tcl's arithmetic fails the build instead of becoming a baked-in bug. Every differential suite reports a skip when no tclsh (or tclsh9.0 / tclsh8.6) is on PATH, so a machine without Tcl still runs the rest.
| What is compared | How |
|---|---|
The twelve syntax rules of Tcl(n) | Rule by rule, against the parser's own structures. |
| Word splitting | Character for character: each case is handed to a Tcl proc that prints its arguments separated by an ASCII record separator, and whatever tclsh reports is the expected value. |
| Whole programs | Byte for byte: each is executed by both implementations and the output compared. Covers the expression and control-flow surface, the list commands, the associative commands, the string ensemble, procedures, and coroutines. |
| Generated matrices | Awkward element values through every list command, foreach through every shape its grammar allows, the glob matcher over a pattern × subject grid, and every index form against lists of every length — each run as one script and compared line for line. A further set checks that each diagnostic matches the interpreter's wording character for character. |
| Interpreter state | Variables, arrays and unset surviving between evaluations, the chunk cache's hit and miss counts, and nesting bounded by an error rather than a stack overflow. |
| The binary | stdout, stderr and exit status against the reference process, in each of its input modes. |
| Ahead-of-time codegen | Every program run both ways and compared byte for byte, including the failing ones, so native codegen cannot diverge from the interpreter. |
The execution corpus covers assignment and substitution, values that must stay strings even though they look numeric (05, 1.10), floored integer division and remainder in all four sign combinations, integral and floating **, double formatting at the positional/exponential boundary, numeric-preferring versus always-string comparison, short-circuit evaluation, the ternary, nested command substitution, and the control-flow commands — plus an unsupported construct being refused rather than approximated, and an i64 overflow promoting to arbitrary precision rather than wrapping — on the interpreted, JIT-compiled and ahead-of-time paths alike.
The list corpus is where the reference implementation earns its title. Canonical quoting is not what the manual implies — an element needing protection only because of a ] or an internal " has those characters escaped while its braces are left bare, so list {a]b} is a\]b — and lsort -unique keeps whichever of two equal elements the reference merge sort happens to keep. Neither could have been guessed; both are pinned by output tclsh produced.
Dependency posture
Dependencies are kept foundational and durable — the goal is a crate that still builds cleanly years from now. The language itself has one: everything the parser, the compiler and the runtime need is fusevm. The rest arrived with the tooling, and a Cargo [[bin]] shares its package's dependency list, so the two that serve only the terminal are declared here too.
| Crate | Role | Reached by |
|---|---|---|
| fusevm | Language-agnostic bytecode VM, 0.26.0, with jit, jit-disk-cache, aot and ffi | the library |
| lsp-server, lsp-types | The framing and the schema of --lsp, so a capability this crate does not answer is a compile error rather than a message an editor silently drops | the library (src/lsp.rs) |
| serde, serde_json | The wire format under both editor servers | the library (src/lsp.rs, src/dap.rs) |
| libc | Capturing a debugged script's stdout, and TIOCGWINSZ plus local time for the prompt's status bar | the library and the binary |
| reedline, nu-ansi-term | The REPL's line editor — history, completion menu, emacs/vi keymaps, multi-line editing — and the colors of its status bar | the binary only |
The test suites add nothing: they shell out to the system tclsh through std::process::Command rather than pulling a harness crate.
Compatibility & longevity
Reference semantics
Behaviour is ported from tclsh 9.0.4 where the slice is implemented, and pinned by suites that diff against the reference interpreter rather than against hand-written expectations.
Refuse, never approximate
Every unimplemented construct is a compile-time error with a Tcl-shaped message and a line number. Nothing silently mis-runs, so the gap list stays honest.
Cross-architecture
Portable bytecode underneath; the Cranelift tiers cover macOS aarch64 and Linux x86_64 / aarch64. Ahead-of-time compilation targets the host only.
Standalone crate
An explicit empty [workspace] keeps tclrs buildable on its own, independent of the meta repo.
Links
- Docs — index.html
- Known gaps — BUGS.md
- Source — github.com/MenkeTechnologies/tclrs
- Issues — github.com/MenkeTechnologies/tclrs/issues
- fusevm — github.com/MenkeTechnologies/fusevm (the shared VM)
- License — MIT (LICENSE)