>_ENGINEERING REPORT
pythonrs is a compiled Python runtime in Rust, hosted on the fusevm bytecode VM and its three-tier Cranelift JIT. This report describes the architecture, the value model, the current state of the tree, and the dependency posture. The statements below are facts about the design and the manifest, not aspirational metrics.
Summary
CPython compiles Python to its own bytecode and walks it in C. pythonrs takes a different path: it lexes and parses Python 3 to an AST, lowers that to fusevm bytecode, and runs it on a compiled VM with a Cranelift JIT. Arithmetic and comparison operators lower to native VM ops so the JIT can trace hot loops; Python-specific behaviour is served by a thread-local PyHost object heap through a numbered builtin-call protocol. It joins fusevm alongside zshrs, stryke, awkrs, elisp, and rubylang, and reuses that shared bytecode VM and JIT rather than shipping its own.
Two things set it apart from every other standalone Python: it transparently caches compiled bytecode via rkyv on every run (~/.pythonrs/scripts.rkyv; a hit skips lex/parse/lower), and it AOT-compiles a script to a standalone native executable via python --build. To our knowledge it is the first compiled standalone Python runtime to do both.
The current tree implements the lexer/parser, AST-to-bytecode lowering, the object heap and method dispatch (str, list, dict, tuple, set, instances), a growing Kernel-builtin and per-type-method surface, functions/closures/*args/**kwargs, user-defined classes with inheritance and operator dunders, list/dict/set comprehensions, f-strings, exceptions (try/except/finally/raise, with), arbitrary-precision int, the standalone python binary and REPL, the transparent rkyv bytecode cache, AOT native-executable emission, and a glob-matched AOP method-intercept registry. Behaviour is checked against the reference python3 by a differential parity harness. The LSP and DAP surfaces are partial; generators, async, match, and most of the standard library are planned — see BUGS.md for the honest ledger.
Hosting on fusevm
pythonrs contains no virtual machine or JIT of its own. The execution path is:
Python source → lexer → parser → AST → lower to fusevm bytecode → fusevm VM + Cranelift JIT
│
PyHost heap (str/list/dict/tuple/set/instances, CallBuiltin)
Shared engine
fusevm is pulled from crates.io with the jit, jit-disk-cache, and aot features. JIT and VM improvements land once and benefit zshrs, stryke, awkrs, elisp, rubylang, and pythonrs together.
Native arithmetic
Operators lower to native fusevm ops; a strict numeric hook supplies Python semantics (str/list concatenation, truthiness, bignum promotion) only for the non-numeric operand cases the VM can't compute natively.
Persistent native code
jit-disk-cache persists compiled machine code across runs, so warm starts skip recompilation. Distinct from the rkyv bytecode cache, which persists the lowered chunks.
AOT emission
The aot feature drives python --build: fusevm::aot emits a native object linked against the pythonrs runtime staticlib, producing a self-contained executable with no interpreter present (src/aot_native.rs).
Value model
Immediate values — int (within i64 range), float, True/False, and None — ride through the VM directly as fusevm::Value. The reference types (str, list, dict, tuple, set, and class instances) live on the PyHost object heap behind Value::Obj(u32) handles, so mutation is observed through every alias — a.append(x) changes the list the caller still holds, exactly as in Python. dict preserves insertion order (an IndexMap), matching CPython's guaranteed iteration order since 3.7. int promotes to arbitrary precision for + - * **.
Truthiness follows Python, not C or the shell: None, False, 0, empty str/list/dict are falsy. Conditions are therefore normalized through a truthiness host op before a native branch, rather than relying on the VM's default numeric truthiness. Operator overloading routes through the instance dunders (__add__, __eq__, __getitem__, __len__, __iter__, …) with reflected __r*__ fallbacks.
Component status
| Component | State | Notes |
|---|---|---|
| lexer / parser → AST | Implemented | Indentation-significant tokenizer (INDENT/DEDENT/NEWLINE), f-strings (src/lexer.rs, src/parser.rs). |
| AST → fusevm bytecode | Implemented | Native arithmetic + CallBuiltin host dispatch; no local VM (src/compiler.rs). |
| object heap & dispatch | Implemented | str/list/dict/tuple/set/instances (src/host.rs, src/builtins.rs). |
| functions / closures / splat / kwargs | Implemented | Def-site *args/**kwargs, defaults, keyword args, real closures (src/compiler.rs). |
| classes / inheritance / operator dunders | Implemented | __init__, instance attrs, arithmetic/comparison dunders with reflected fallbacks, __getitem__/__len__/__iter__/__repr__. |
| comprehensions / f-strings | Implemented | list/dict/set comprehensions; f-string format mini-language (fill/align/sign/width/,/.prec/type). |
exceptions / with | Implemented | try/except/finally/raise, typed classes; with desugars to try/finally over __enter__/__exit__. |
arbitrary-precision int | Implemented | Bignum promotion for + - * **; ///%/bitwise stay i64-range (see BUGS). |
standalone python binary + REPL | Implemented | Files, -c one-liners, interactive REPL (src/main.rs, src/repl.rs). |
| transparent rkyv bytecode cache | Implemented | Versioned shard at ~/.pythonrs/scripts.rkyv, consulted on every run (src/cache.rs). |
AOT native executable (--build) | Implemented | fusevm::aot emission linked to the runtime staticlib (src/aot_native.rs). |
| AOP method intercepts | Implemented | Glob-matched before/after/around registry (src/intercepts.rs). |
parity harness vs reference python3 | Implemented | Differential corpus diffed byte-for-byte against CPython (src/bin/parity.rs). |
LSP server (--lsp) | Partial | Initialize/shutdown handshake, builtin/keyword completion, stub hover; no diagnostics/go-to-def yet (src/lsp.rs). |
DAP adapter (--dap) | Partial | Per-statement line-marker compile mode exists; the stepping server returns "not implemented" (src/dap.rs). |
| generators / async / match / stdlib | Planned | yield, await, structural pattern matching, and most of the standard library beyond minimal math/sys pending. |
Dependency posture
Dependencies are kept foundational and durable — the goal is a crate that still builds cleanly years from now. Direct dependencies from Cargo.toml:
| Crate | Role |
|---|---|
| fusevm | Language-agnostic bytecode VM + three-tier Cranelift JIT (jit, jit-disk-cache, aot) |
| clap | CLI argument parsing (derive, cargo) |
| thiserror | Error type derivation |
| rustc-hash | Fast hashing for the script-cache key |
| indexmap | Insertion-ordered dict storage (matches CPython iteration) |
| rkyv / bincode | Zero-copy outer shard + serde inner blobs for the transparent bytecode cache |
| memmap2 / dirs / once_cell | Cache file access + platform cache directory + lazy statics |
| lsp-server / lsp-types | LSP transport + protocol types (--lsp server) |
| serde / serde_json / libc | DAP/LSP JSON-RPC plumbing; libc pipe + dup2 to capture debuggee stdout |
| reedline / nu-ansi-term / sysinfo | Interactive REPL line editor, colors, live stats |
| glob | Pattern matching for the AOP method-intercept engine |
Compatibility & longevity
Versioned caches
The rkyv script-cache format carries a schema version from the first release, so an old cached .py rebuilds silently rather than breaking.
Reference semantics
Behaviour tracks CPython — integer // floors, dict preserves insertion order, aliases share mutation — as the compatibility target, verified byte-for-byte on the parity corpus.
Cross-architecture
macOS aarch64 and Linux x86_64 / aarch64 via the Cranelift JIT; portable bytecode underneath.
Standalone crate
An explicit empty [workspace] keeps pythonrs buildable on its own, independent of the meta repo.
Links
- Docs — index.html
- Source — github.com/MenkeTechnologies/pythonrs
- Issues — github.com/MenkeTechnologies/pythonrs/issues
- fusevm — github.com/MenkeTechnologies/fusevm (the shared VM)
- License — MIT (LICENSE)