// PYTHONRS — ENGINEERING REPORT

pythonrs v0.1.10 · Python on fusevm · lex/parse → AST → bytecode → Cranelift JIT · transparent rkyv cache on every run · AOT native-exe · MIT · in active development

Docs GitHub
// Color scheme

>_ENGINEERING REPORT

pythonrs is a compiled Python runtime in Rust, hosted on the fusevm bytecode VM and its three-tier Cranelift JIT. This report describes the architecture, the value model, the current state of the tree, and the dependency posture. The statements below are facts about the design and the manifest, not aspirational metrics.

Python
fusevm language host
3
Cranelift JIT tiers
v0.1.10
version
MIT
license · free / OSS

Summary

CPython compiles Python to its own bytecode and walks it in C. pythonrs takes a different path: it lexes and parses Python 3 to an AST, lowers that to fusevm bytecode, and runs it on a compiled VM with a Cranelift JIT. Arithmetic and comparison operators lower to native VM ops so the JIT can trace hot loops; Python-specific behaviour is served by a thread-local PyHost object heap through a numbered builtin-call protocol. It joins fusevm alongside zshrs, stryke, awkrs, elisp, and rubylang, and reuses that shared bytecode VM and JIT rather than shipping its own.

Two things set it apart from every other standalone Python: it transparently caches compiled bytecode via rkyv on every run (~/.pythonrs/scripts.rkyv; a hit skips lex/parse/lower), and it AOT-compiles a script to a standalone native executable via python --build. To our knowledge it is the first compiled standalone Python runtime to do both.

The current tree implements the lexer/parser, AST-to-bytecode lowering, the object heap and method dispatch (str, list, dict, tuple, set, instances), a growing Kernel-builtin and per-type-method surface, functions/closures/*args/**kwargs, user-defined classes with inheritance and operator dunders, list/dict/set comprehensions, f-strings, exceptions (try/except/finally/raise, with), arbitrary-precision int, generators (yield / yield from), async/await on a native event loop, match/case structural pattern matching, the standalone python binary and REPL, the transparent rkyv bytecode cache, AOT native-executable emission, and a glob-matched AOP method-intercept registry. The standard library arrives through the stdlib-ffi bridge to an embedded libpython (on by default). Behaviour is checked against the reference python3 by a differential parity harness and a differential fuzzer. The LSP and DAP surfaces are partial — see BUGS.md for the honest ledger.


Hosting on fusevm

pythonrs contains no virtual machine or JIT of its own. The execution path is:

Python source  →  lexer  →  parser → AST  →  lower to fusevm bytecode  →  fusevm VM + Cranelift JIT
                                                    │
                                       PyHost heap (str/list/dict/tuple/set/instances, CallBuiltin)

Shared engine

fusevm is pulled from crates.io with the jit, jit-disk-cache, and aot features. JIT and VM improvements land once and benefit zshrs, stryke, awkrs, elisp, rubylang, and pythonrs together.

Native arithmetic

Operators lower to native fusevm ops; a strict numeric hook supplies Python semantics (str/list concatenation, truthiness, bignum promotion) only for the non-numeric operand cases the VM can't compute natively.

Persistent native code

jit-disk-cache persists compiled machine code across runs, so warm starts skip recompilation. Distinct from the rkyv bytecode cache, which persists the lowered chunks.

AOT emission

The aot feature drives python --build: fusevm::aot emits a native object linked against the pythonrs runtime staticlib, producing a self-contained executable with no interpreter present (src/aot_native.rs).


Value model

Immediate values — int (within i64 range), float, True/False, and None — ride through the VM directly as fusevm::Value. The reference types (str, list, dict, tuple, set, and class instances) live on the PyHost object heap behind Value::Obj(u32) handles, so mutation is observed through every alias — a.append(x) changes the list the caller still holds, exactly as in Python. dict preserves insertion order (an IndexMap), matching CPython's guaranteed iteration order since 3.7. int promotes to arbitrary precision across + - * ** // % and the bitwise ops & | ^ << >>.

Truthiness follows Python, not C or the shell: None, False, 0, empty str/list/dict are falsy. Conditions are therefore normalized through a truthiness host op before a native branch, rather than relying on the VM's default numeric truthiness. Operator overloading routes through the instance dunders (__add__, __eq__, __getitem__, __len__, __iter__, …) with reflected __r*__ fallbacks.


Component status

ComponentStateNotes
lexer / parser → ASTImplementedIndentation-significant tokenizer (INDENT/DEDENT/NEWLINE), f-strings (src/lexer.rs, src/parser.rs).
AST → fusevm bytecodeImplementedNative arithmetic + CallBuiltin host dispatch; no local VM (src/compiler.rs).
object heap & dispatchImplementedstr/list/dict/tuple/set/instances (src/host.rs, src/builtins.rs).
functions / closures / splat / kwargsImplementedDef-site *args/**kwargs, defaults, keyword args, real closures, and call-site */** unpacking (src/compiler.rs).
classes / inheritance / operator dundersImplemented__init__, instance attrs, arithmetic/comparison dunders with reflected fallbacks, __getitem__/__len__/__iter__/__repr__.
comprehensions / f-stringsImplementedlist/dict/set comprehensions; f-string format mini-language (fill/align/sign/width/,/.prec/type).
exceptions / withImplementedtry/except/finally/raise, typed classes; with desugars to try/finally over __enter__/__exit__.
arbitrary-precision intImplementedBignum promotion across + - * ** // % and the bitwise ops & | ^ << >> — the earlier i64 cap is gone.
standalone python binary + REPLImplementedFiles, -c one-liners, interactive REPL (src/main.rs, src/repl.rs).
transparent rkyv bytecode cacheImplementedVersioned shard at ~/.pythonrs/scripts.rkyv, consulted on every run (src/cache.rs).
AOT native executable (--build)Implementedfusevm::aot emission linked to the runtime staticlib (src/aot_native.rs). Requires a --no-default-features build; a stdlib-ffi binary cannot statically link CPython.
AOP method interceptsImplementedGlob-matched before/after/around registry (src/intercepts.rs).
parity harness vs reference python3ImplementedDifferential corpus diffed byte-for-byte against CPython (src/bin/parity.rs).
generators / async / matchImplementedStackful corosensei generators with full yield from, a native fusevm asyncio event loop, and PEP 634 structural pattern matching.
diagnostics (--doctor / --cacheview / --tiers)ImplementedRuntime/CPython/cache/env report, rkyv-shard listing, and a per-chunk fusevm execution-tier report (src/tiers.rs).
LSP server (--lsp)PartialBuiltin/keyword/method completion, position-aware hover, and diagnostics from the real parser; go-to-def and signature help not yet (src/lsp.rs).
DAP adapter (--dap)PartialBreakpoints, step in/out/over/continue, stack trace, locals, expression evaluate, program-stdout capture; watch expressions not yet (src/dap.rs).
standard libraryPartialThe stdlib-ffi bridge (default) serves the real CPython stdlib over an embedded libpython; sys stays wholly native and math/collections/functools/contextlib resolve native arms first; --no-default-features drops the bridge for the vendored pylib/ tree.

Dependency posture

Dependencies are kept foundational and durable — the goal is a crate that still builds cleanly years from now. Direct dependencies from Cargo.toml:

CrateRole
fusevmLanguage-agnostic bytecode VM + three-tier Cranelift JIT (jit, jit-disk-cache, aot, ffi)
clapCLI argument parsing (derive, cargo)
thiserrorError type derivation
rustc-hashFast hashing for the script-cache key
indexmapInsertion-ordered dict storage (matches CPython iteration)
rkyv / bincodeZero-copy outer shard + serde inner blobs for the transparent bytecode cache
memmap2 / dirs / once_cellCache file access + platform cache directory + lazy statics
lsp-server / lsp-typesLSP transport + protocol types (--lsp server)
serde / serde_json / libcDAP/LSP JSON-RPC plumbing; libc pipe + dup2 to capture debuggee stdout
reedline / nu-ansi-term / sysinfoInteractive REPL line editor, colors, live stats
globPattern matching for the AOP method-intercept engine
corosenseiStackful same-thread coroutines backing generators and the async event loop
num-bigint / num-integer / num-traitsArbitrary-precision int beyond the i64 fast path
num-rational / num-complexExact rationals for fractions.Fraction; complex support
libmPure-Rust special functions std lacks (lgamma/gamma/erf/erfc) — no C libm
regex / fancy-regexThe re engines: linear-time for the majority, look-around/backreferences for the rest
digest / md-5 / sha1 / sha2 / sha3 / blake2hashlib backends — audited implementations of the same FIPS/RFC specs
unicode_names2 / unicode-general-category\N{…} name lookup and Unicode category queries
pyo3 (optional, on by default)The stdlib-ffi bridge to an embedded libpython (abi3-py39); dropped by --no-default-features

Compatibility & longevity

Versioned caches

The rkyv script-cache format carries a schema version from the first release, so an old cached .py rebuilds silently rather than breaking.

Reference semantics

Behaviour tracks CPython — integer // floors, dict preserves insertion order, aliases share mutation — as the compatibility target, verified byte-for-byte on the parity corpus.

Cross-architecture

macOS aarch64 and Linux x86_64 / aarch64 via the Cranelift JIT; portable bytecode underneath.

Standalone crate

An explicit empty [workspace] keeps pythonrs buildable on its own, independent of the meta repo.


Links