>_ENGINEERING REPORT
texrs is a TeX engine in Rust — Knuth's mouth and expander — hosted on the fusevm bytecode VM and its three-tier Cranelift JIT. This report describes the architecture, the decisions lowering forced, how parity is measured, and the dependency posture. The statements below are facts about the design, the manifest and the test suite, not aspirational metrics.
Summary
Every mainstream TeX engine — pdfTeX, XeTeX, LuaTeX — descends from Knuth's tex.web through web2c, and every one of them interprets the expander. texrs compiles it. Source is tokenised under a mutable category-code table, macros are expanded while lowering, and what is left becomes fusevm bytecode that the shared VM executes and the Cranelift JIT can compile.
The scope is deliberate and stated plainly: this is the mouth, the expander, and a stomach that produces a page with maths, pictures, colour and images on it. --pdf breaks a paragraph by minimising its total demerits over every feasible set of breakpoints and hyphenates with Liang patterns; --dvi takes the first break that fits, because a DVI driver cannot set a run to a width. --pdf breaks its pages by penalty too, over the whole document: widows, orphans, hyphenated lines and stranded headings. Neither has maths or boxes a document can nest, so a paragraph set here and the same paragraph set by tex will not agree line for line. The mouth and the expander are the half of TeX a macro-heavy document spends its time in, and the half where compilation has something to prove.
Hosting on fusevm
texrs carries no VM and no JIT of its own. It is a frontend, in the same sense zshrs, stryke, rubylang, pythonrs and scalars are frontends:
TeX source → mouth → expander → command stream → fusevm bytecode → VM + Cranelift JIT
Registers are slots
TeX has exactly 256 \count registers (tex.web §236). They map onto VM slots 0..255, so a register read is an array index and \advance\count0 by 5 is GetSlot / LoadInt / Add / SetSlot — ops the JIT can compile, not a hash lookup and a match arm.
Conditionals are branches
\ifnum tests run-time state, so it lowers to a comparison plus a jump. Both arms are collected as token runs and lowered recursively. A tree-walker cannot do this: it decides the branch as it walks.
Constant folding
\iftrue and \ifx depend only on the macro table, which lowering already owns — so they are decided at compile time and never reach the VM. tests/lowering.rs asserts the emitted bytecode, because output parity alone would not distinguish a frontend from a tree-walker.
Bytecode cache
What a document compiled to is kept in an rkyv shard — the layout the sibling engines write, under texrs's own magic — keyed by path and valid while the source's mtime matches to the nanosecond. A hit skips the mouth, the expander and the lowerer entirely. Writes take a lock, fsync and rename, so a reader never sees a half-written shard.
Two host builtins
\message is an append and a flush on the host side. That is what makes a message containing \the\count0 read the register at run time rather than freezing it when the file was read.
What lowering forces
Two categories of work cannot be deferred, and one cannot be hoisted:
| Construct | When it happens | Why |
|---|---|---|
\catcode | Compile time | It changes how the rest of the file reads. Deferring it would mean tokenising text whose categories are not known yet. |
\def, \let, \edef | Compile time | The macro table decides what the following tokens mean; expansion is a source-to-source transform, not a run-time call. |
\count assignment, arithmetic | Run time | A register is a slot the program writes. \advance lowers to native ops. |
\ifnum, \ifodd, \ifcase | Run time (real branch) | They test registers, which only exist while the program runs. |
\iftrue, \iffalse, \ifx | Compile time (folded) | Decidable from the macro table alone; emitting a branch would be dead weight. |
| Brace group | Both | It scopes the macro table (compile time) and the registers written inside it (a save/restore pair around the lowered body). |
Parity posture
The contract for this milestone is the \message stream, compared byte-for-byte against the real tex binary. Nothing is written by hand: the reference is produced by running tex at test time, so a case cannot be made to pass by editing an expectation.
| Harness | What it does |
|---|---|
tests/differential.rs | Every case in tests/cases against real tex. Fails on any divergence not listed in tests/known_gaps.txt, and on a listed case that has started passing — so the list cannot go stale. |
tests/examples.rs | Every program in examples/, held to the same comparison with no known-gap escape hatch. Documentation that has drifted from the engine is worse than none. |
tests/lowering.rs | Asserts the emitted bytecode: that a conditional became a branch, that \count arithmetic became native ops, that a foldable conditional was folded. |
the parity-fuzz binary | Generates seeded random programs confined to the implemented subset, runs both engines in parallel, and reduces whatever diverges — dropping statements while the divergence survives, refusing a reduction that turns it into a different divergence. |
tests/fuzz_smoke.rs, tests/fuzz_mass_replay.rs | Replays the cargo-fuzz targets on their seed corpus under stable Rust, and points the mouth at generated mutations of every .tex in the tree. |
fuzz/ | cargo-fuzz targets lex, lower and run, looking for panics rather than divergences. |
The oracle itself is pinned: both shell harnesses read the tex version out of BUGS.md and refuse a different engine. A mismatched oracle does not fail loudly — it reports a different divergence set, which reads exactly like a regression.
The fuzzer has already paid for itself: it found that \edef does not freeze a conditional in its body, that an undefined control sequence prints its own name instead of raising, and a panic in the argument reader on TeX's #{ parameter form. Each is a committed case with its reason written down.
Dependency posture
One dependency: fusevm, with the jit and jit-disk-cache features. No parser generator, no regex engine, no async runtime, no C shim. The mouth and the expander are hand-written because tex.web is the specification and a generated parser cannot express a category-code table the document mutates while it is being read.
That posture is a durability decision as much as a performance one. The crate should still build from a fresh checkout in ten years; every added dependency is a bet that someone else maintains theirs for that long.
Gaps, stated plainly
BUGS.md is the ledger, and every entry in it is pinned by a case the test suite gates on. In summary: a stomach that stops short of tex's — --pdf breaks by total demerits and hyphenates, --dvi is still first-fit, and neither breaks pages by penalties, so byte-identical DVI parity is not approached; no registers other than \count; no format preloaded, so registers start at INITEX zero while the reference tex has plain's values; an undefined control sequence prints its name instead of raising; \edef does not decide a conditional at definition time; #{ parameter text is refused; and characters above U+00FF are one token here where TeX82 reads bytes.
Links
- Documentation — index.html
- Primitive reference — reference.html
- Source — github.com/MenkeTechnologies/texrs
- The shared VM — fusevm