Metal boots into a persistent native MicroPython REPL

[pymergetic]

First of all: thank you to the MicroPython community and maintainers. MicroPython’s embeddability is what made this experiment practical in the first place.

I’m building Metal, an experimental UEFI/BIOS freestanding async runtime. It is not a conventional OS and it is very far from production-ready, but it now boots directly into a persistent interactive MicroPython REPL.

The important distinction is that MicroPython is native core code in Metal — not a Wasm guest and not a fresh interpreter created for each command. py <script> and py -c schedule more work on the same long-lived runtime; the C command console remains available through console().

Some details of the current experiment:

bidirectional host bindings: Python calls Metal/C, and Metal/C can call Python callables;
a persistent shared context for the normal path, with globals kept alive;
Python tasks and coroutines share Metal’s async scheduling model;
opt-in isolated contexts with their own heap and GC nursery for genuinely parallel bytecode execution;
a signed, trust-checked stdlib.zip with pure-Python modules and Metal-specific bindings;
WAMR remains a separate execution layer for signed Wasm/AOT mods — MicroPython is not being run through WAMR.
I’ve attached a boot/REPL screenshot. The pmcmd.ping("127.0.0.1") example is deliberately mundane, but it shows the Python side talking to a live Metal host/network stack rather than just printing a hello-world.

Repo: https://github.com/pymergetic/metal

I’m posting this primarily to say thanks and to ask for criticism, not applause. If you know the MicroPython port layer, scheduler, GC, heap/lifetime model, or long-lived embedding patterns: what would you question or redesign first? I would especially value feedback on the shared-runtime versus isolated-context model, async host bindings, and invariants this design must not violate.

[farfromexact]

The shared-runtime versus isolated-context choice is the part I would scrutinize first. MicroPython's VM, heap, scheduler, and C object lifetimes form one coupled system; adding concurrent host tasks does not automatically make a single interpreter re-entrant.

The invariants I would make explicit are:

• only one execution context enters a given VM at a time unless the port adds a real interpreter lock;
• every mp_obj_t retained by a C callback is rooted for the entire callback lifetime;
• no pointer into a movable or collectible heap is kept by Metal after a safe point;
• a context cannot free or mutate another context's heap;
• cancellation cannot run arbitrary Python while the VM or host bridge holds a non-reentrant lock;
• callbacks into Python are serialized at defined scheduler safe points.

For the shared mode, a single event-loop owner with queued callbacks is safer than allowing arbitrary Metal threads to call Python. For isolated contexts, give each interpreter its own heap, scheduler state, exception state, and module cache, and define how values cross the boundary. Copy/serialize values first; do not pass raw Python object references between heaps.

The signed stdlib archive provides integrity, not isolation. Host bindings are the actual capability surface, so they need an explicit lifetime and permission model. I would test allocation pressure, nested callbacks, cancellation during I/O, GC during a host callback, and an exception crossing the C/Python boundary before optimizing parallel execution.

A persistent REPL is a great demonstration, but the scheduler/GC ownership rules should be written down before third-party native bindings depend on them.

[pymergetic]

Thank you — this is exactly the kind of review I was hoping for. As said, I need to go back to the drawing board and make the scheduler, GC, object-lifetime, and bridge ownership rules explicit before taking the parallel-context idea any further.

Metal already has a multi-module loader with explicit SINGLE/MULTI capability and fresh WAMR instances for multi-capable modules. The host-side model I have in mind is closer to a conveyor belt of explicit work packages than shared task state: a parent operation owns a package’s lifetime, and child work receives only the input it is allowed to process — ideally an immutable snapshot or an exclusively transferred handle. Results return as new packages instead of children mutating a shared object.

I would prefer not to use locks as a substitute for unclear ownership. The goal is to keep most work stateless, with only a small explicit continuation/handle state surviving a yield; each next step receives the state it needs rather than reaching back into shared mutable task data.

My current assumption is that the particular CPU running a child should not matter as long as ownership, publication/memory ordering, and lifetime are explicit: the parent must not free or reuse input before children have completed and their results have been gathered. Is that a reasonable framing, or am I still missing an important failure mode?

But I agree this remains separate from runtime safety. The existing task graph and isolated guest instances can use that ownership model in parallel, while the persistent MicroPython runtime needs its own serialized entry rule: one explicit VM owner and callbacks queued at scheduler-safe points.

You are also right that the signed stdlib archive provides integrity and provenance, not isolation. Metal’s current model deliberately trusts first-party guest code; it is not trying to treat independently authored or hostile Wasm modules as mutually untrusted tenants.

The intended boundary is closer to an integrated appliance: a minimal runtime plus applications built and released through one controlled pipeline. Whether an application is baked into the image or delivered later over HTTP, the kernel should accept only artifacts from that signed first-party release chain. The aim is that the host, runtime, stdlib, and application arrive as one coherent system rather than as arbitrary third-party code loaded at runtime. Keeping the system small helps reduce the trusted computing base and makes that release boundary easier to reason about, but it does not replace guest isolation.

If Metal ever needs to run independently supplied or hostile modules, that would require a separate isolation and capability/security model. I’ll write the invariants down and test the failure cases you listed — allocation pressure, nested callbacks, cancellation during I/O, GC during host callbacks, and exceptions across the C/Python boundary — before presenting the current model as anything more than an experiment. Thanks again; this gives me a much better checklist for the next design pass.