riscv32im-unknown-openvm-elf: add target

[arayikhalatyan]

This pull request adds OpenVM as a new target for Rust. The riscv32im-unknown-openvm-elf target runs inside a virtual machine with full support for the RV32IM ISA. The target includes limited support for std through customizable non-standard RISC-V instruction set extensions which are configurable in the OpenVM framework. While std library support is currently limited (see book for details), it is important to have this functionality for downstream users to compile existing crates that are not no_std but in practice require limited OS functionality. Moreover, additional support for more std features is expected and can be customizable through extensions of the VM.

We have chosen the -unknown-openvm-elf naming policy in accordance with the discussion in #135376 (comment)

Further details may be found in the target description doc.

Tier 3 target policy:

Here is a copy of the tier 3 target policy:

Tier 3 target policy:
At this tier, the Rust project provides no official support for a target, so we
place minimal requirements on the introduction of targets.
A proposed new tier 3 target must be reviewed and approved by a member of the
compiler team based on these requirements. The reviewer may choose to gauge
broader compiler team consensus via a [Major Change Proposal (MCP)](https://forge.rust-lang.org/compiler/mcp.html).
A proposed target or target-specific patch that substantially changes code
shared with other targets (not just target-specific code) must be reviewed and
approved by the appropriate team for that shared code before acceptance.

• A tier 3 target must have a designated developer or developers (the "target
  maintainers") on record to be CCed when issues arise regarding the target.
  (The mechanism to track and CC such developers may evolve over time.)

The maintainers are named in the target description file

• Targets must use naming consistent with any existing targets; for instance, a
  target for the same CPU or OS as an existing Rust target should use the same
  name for that CPU or OS. Targets should normally use the same names and
  naming conventions as used elsewhere in the broader ecosystem beyond Rust
  (such as in other toolchains), unless they have a very good reason to
  diverge. Changing the name of a target can be highly disruptive, especially
  once the target reaches a higher tier, so getting the name right is important
  even for a tier 3 target.
• Target names should not introduce undue confusion or ambiguity unless
  absolutely necessary to maintain ecosystem compatibility. For example, if
  the name of the target makes people extremely likely to form incorrect
  beliefs about what it targets, the name should be changed or augmented to
  disambiguate it.
• If possible, use only letters, numbers, dashes and underscores for the name.
  Periods (.) are known to cause issues in Cargo.

We understand. While the target shares similarities to riscv32im-risc0-zkvm-elf, we choose the riscv32im-unknown-openvm-elf naming in accordance with the discussion in #135376 (comment)

• Tier 3 targets may have unusual requirements to build or use, but must not
  create legal issues or impose onerous legal terms for the Rust project or for
  Rust developers or users.

  • The target must not introduce license incompatibilities.

We understand and will not introduce incompatibilities.

• Anything added to the Rust repository must be under the standard Rust license (MIT OR Apache-2.0).

We understand. The code added to the Rust repository is under MIT OR Apache-2.0 license.

• The target must not cause the Rust tools or libraries built for any other
  host (even when supporting cross-compilation to the target) to depend
  on any new dependency less permissive than the Rust licensing policy. This
  applies whether the dependency is a Rust crate that would require adding
  new license exceptions (as specified by the tidy tool in the
  rust-lang/rust repository), or whether the dependency is a native library
  or binary. In other words, the introduction of the target must not cause a
  user installing or running a version of Rust or the Rust tools to be
  subject to any new license requirements.

We understand. The runtime libraries and the execution environment and software associated with this environment uses MIT OR Apache-2.0 so this should not be an issue.

• Compiling, linking, and emitting functional binaries, libraries, or other
  code for the target (whether hosted on the target itself or cross-compiling
  from another target) must not depend on proprietary (non-FOSS) libraries.
  Host tools built for the target itself may depend on the ordinary runtime
  libraries supplied by the platform and commonly used by other applications
  built for the target, but those libraries must not be required for code
  generation for the target; cross-compilation to the target must not require
  such libraries at all. For instance, rustc built for the target may
  depend on a common proprietary C runtime library or console output library,
  but must not depend on a proprietary code generation library or code
  optimization library. Rust's license permits such combinations, but the
  Rust project has no interest in maintaining such combinations within the
  scope of Rust itself, even at tier 3.

We understand. All OpenVM crates that may be required for correct linking are also licensed under MIT OR Apache-2.0 license.

• "onerous" here is an intentionally subjective term. At a minimum, "onerous"
  legal/licensing terms include but are not limited to: non-disclosure
  requirements, non-compete requirements, contributor license agreements
  (CLAs) or equivalent, "non-commercial"/"research-only"/etc terms,
  requirements conditional on the employer or employment of any particular
  Rust developers, revocable terms, any requirements that create liability
  for the Rust project or its developers or users, or any requirements that
  adversely affect the livelihood or prospects of the Rust project or its
  developers or users.

There are no such terms present

• Neither this policy nor any decisions made regarding targets shall create any
  binding agreement or estoppel by any party. If any member of an approving
  Rust team serves as one of the maintainers of a target, or has any legal or
  employment requirement (explicit or implicit) that might affect their
  decisions regarding a target, they must recuse themselves from any approval
  decisions regarding the target's tier status, though they may otherwise
  participate in discussions.

We understand.

• This requirement does not prevent part or all of this policy from being
  cited in an explicit contract or work agreement (e.g. to implement or
  maintain support for a target). This requirement exists to ensure that a
  developer or team responsible for reviewing and approving a target does not
  face any legal threats or obligations that would prevent them from freely
  exercising their judgment in such approval, even if such judgment involves
  subjective matters or goes beyond the letter of these requirements.

We understand.

• Tier 3 targets should attempt to implement as much of the standard libraries
  as possible and appropriate (core for most targets, alloc for targets
  that can support dynamic memory allocation, std for targets with an
  operating system or equivalent layer of system-provided functionality), but
  may leave some code unimplemented (either unavailable or stubbed out as
  appropriate), whether because the target makes it impossible to implement or
  challenging to implement. The authors of pull requests are not obligated to
  avoid calling any portions of the standard library on the basis of a tier 3
  target not implementing those portions.

The target implements core and alloc. Support for std is limited, with some functionality stubbed out. The target provides std support through custom RISC-V instruction set extensions (see book for details) and is not meant to provide a full OS. However std support is still important for downstream users of the target due to the need to compile existing crates that are not no_std but in practice require limited OS functionality. Furthermore, due to the customizable framework of the target virtual machines, more comprehensive std support may be implemented with extensions in the future.

• The target must provide documentation for the Rust community explaining how
  to build for the target, using cross-compilation if possible. If the target
  supports running binaries, or running tests (even if they do not pass), the
  documentation must explain how to run such binaries or tests for the target,
  using emulation if possible or dedicated hardware if necessary.

See file target description file

• Tier 3 targets must not impose burden on the authors of pull requests, or
  other developers in the community, to maintain the target. In particular,
  do not post comments (automated or manual) on a PR that derail or suggest a
  block on the PR based on a tier 3 target. Do not send automated messages or
  notifications (via any medium, including via @) to a PR author or others
  involved with a PR regarding a tier 3 target, unless they have opted into
  such messages.

We understand.

• Backlinks such as those generated by the issue/PR tracker when linking to
  an issue or PR are not considered a violation of this policy, within
  reason. However, such messages (even on a separate repository) must not
  generate notifications to anyone involved with a PR who has not requested
  such notifications.

We understand.

• Patches adding or updating tier 3 targets must not break any existing tier 2
  or tier 1 target, and must not knowingly break another tier 3 target without
  approval of either the compiler team or the maintainers of the other tier 3
  target.

  • In particular, this may come up when working on closely related targets,
    such as variations of the same architecture with different features. Avoid
    introducing unconditional uses of features that another variation of the
    target may not have; use conditional compilation or runtime detection, as
    appropriate, to let each target run code supported by that target.

We understand.

Tier 3 targets must be able to produce assembly using at least one of rustc's supported backends from any host target. (Having support in a fork of the backend is not sufficient, it must be upstream.)

ELF binaries for the target can be generated using the LLVM backend.

If a tier 3 target stops meeting these requirements, or the target maintainers
no longer have interest or time, or the target shows no signs of activity and
has not built for some time, or removing the target would improve the quality
of the Rust codebase, we may post a PR to remove it; any such PR will be CCed
to the target maintainers (and potentially other people who have previously
worked on the target), to check potential interest in improving the situation.

We understand.

[rustbot]

The Miri subtree was changed

cc @rust-lang/miri

Some changes occurred in src/doc/rustc/src/platform-support

cc @Noratrieb

These commits modify compiler targets.
(See the Target Tier Policy.)

[rustbot]

r? @ChrisDenton

rustbot has assigned @ChrisDenton.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[rustbot]

⚠️ Warning ⚠️

• There are issue links (such as #123) in the commit messages of the following commits.
  Please move them to the PR description, to avoid spamming the issues with references to the commit, and so this bot can automatically canonicalize them to avoid issues with subtree.
  • 5b5e109

[arayikhalatyan]

r? compiler

[RalfJung]

New targets need an MCP to be filed at https://github.com/rust-lang/compiler-team/issues. I don't think that has happened yet for this one?

The MCP should explain what exactly is meant by "limited" std support. Which operations work, which don't? We want to avoid repeating the mistake that is wasm32-unk-unk where hardly anything in std works -- that target should have been no_std.

[mati865]

MCP is optional for tier 3 targets:

A proposed new tier 3 target must be reviewed and approved by a member of the compiler team based on these requirements. The reviewer may choose to gauge broader compiler team consensus via a Major Change Proposal (MCP).

IIRC, it has changed this year.

[RalfJung]

Ah, I see.

The comment about "limited" std support stands though -- please summarize what exactly that means in terms of which operations are available and which are missing. In particular, are there any operations that just panic because they cannot return a Result? That would be problematic and an indication that maybe we should make this a no_std target instead.

[jieyouxu]

MCP is optional for tier 3 targets:

A proposed new tier 3 target must be reviewed and approved by a member of the compiler team based on these requirements. The reviewer may choose to gauge broader compiler team consensus via a Major Change Proposal (MCP).

IIRC, it has changed this year.

Yes, the intention is that if it's non-controversial, a PR suffices, but the reviewer can elect to request a MCP esp. if the proposed target has quirks ™️ (such as limited std support, or discuss things about a target that possibly can never progress past Tier 3 for whatever reason).

[ChrisDenton]

I believe libs have previously said that they would want new tier 2 std targets to at least support std::time::Instant. However, tier 3 targets have been given some leeway here.

cc @rust-lang/libs-api @rust-lang/libs

[jonathanpwang]

Regarding std, copying this over from the OpenVM book for convenience and extra commentary:

OpenVM supports standard Rust written using the std library, with the following limitations that users should be aware of:

• Standard input (e.g., from console) is not supported. Use the openvm::io::read methods instead.
  • We can support this in the future by supplying the sys_read extern function. It would just passthrough to openvm::io::read but match the specifications of the Read trait.
• Standard output and standard error (e.g., println!, eprintln!) are supported and will both print to the host standard output.
• System randomness calls are supported by default. Important: system randomness requests randomness from the host, and the provided randomness is unvalidated.
  Users must be aware of this and only use system randomness in settings where this meets their security requirements. In particular, system randomness should not be used for cryptographic purposes.
• Reading of environment variables will always return None.
  • This could be supported via passthrough to host. We currently don't support it because it confuses what is done by the guest.
• Reading of argc and argv will always return 0.
  • Same here.

For the current implementations of the PAL abi extern functions, see https://github.com/openvm-org/openvm/blob/main/crates/toolchain/openvm/src/pal_abi.rs

For the unimplemented functionality mentioned, we do not panic since this may cause infinite recursive calls. However we do call a special system instruction that terminates the program.

However as mentioned in the PR description, it is very important that this target is not no_std since the most common use case is still to execute the target on existing Rust crates, many of which do not have a no_std option. Our understanding was that for Tier 3, the level of support for std above is acceptable. We are happy to discuss specific concerns about missing support.

[RalfJung]

OpenVM supports standard Rust written using the std library, with the following limitations that users should be aware of:

Thanks for listing that!
So this list is exhaustive? In particular, what happens when I call Instant::ellapsed or SystemTime::now?

However as mentioned in the PR description, it is very important that this target is not no_std since the most common use case is still to execute the target on existing Rust crates, many of which do not have a no_std option. Our understanding was that for Tier 3, the level of support for std above is acceptable. We are happy to discuss specific concerns about missing support.

Yes, I have seen that. However, for the health of the overall Rust ecosysten, it is also important to avoid repeating the wasm32-u-u mistake.

If what @ChrisDenton wrote is correct, then indeed for tier 3 it doesn't really matter. :)
(Maybe we should downgrade wasm32-u-u to tier 3... but that's a different discussion ;)

[ChrisDenton]

Actually I'm a bit baffled about the need for std. The linked book strongly suggests using no_std. And the std support that is there seems limited to stdout and random.

[ChrisDenton]

Actually I'm a bit baffled about the need for std. The linked book strongly suggests using no_std. And the std support that is there seems limited to stdout and random.

[jonathanpwang]

So this list is exhaustive? In particular, what happens when I call Instant::ellapsed or SystemTime::now?

Not exhaustive as it doesn't cover the cases where we just used the default unsupported modules:

#[path = "../unsupported/os.rs"]
pub mod os;
#[path = "../unsupported/pipe.rs"]
pub mod pipe;
#[path = "../unsupported/time.rs"]
pub mod time;

Based on the implementation there, I believe calling any time operation will panic indeed.

Actually I'm a bit baffled about the need for std.

We recommend no_std for new library development since we do think that is the correct best practice if you're creating a library from scratch. However often that won't be the case and users need to run a binary pulling in many external dependencies. For example, one primary application is running https://github.com/paradigmxyz/reth/tree/main (see https://github.com/axiom-crypto/openvm-reth-benchmark as an example). While very very recently reth does now have no_std support, previously it did not and running the binary with no_std involved many small forks / modifications of dependencies, when in reality the actual std usage was quite minimal.

[RalfJung]

That sounds like exactly the argument that was given for wasm. The result is a mess; many crates are std because they actually need std APIs so they fail in random unexpected ways due to things panicking, and often you have to add explicit cfg to actually have sensible wasm support.

Actually patching dependencies for no_std support is the proper thing to do here -- actually fixing the problem once rather than having everyone deal with a never-ending stream of odd panics and other unexpected failures. It's more work upfront, but the result is a much healthier ecosystem. (Long-term we probably want a more modular approach to std overall, but I assume that will only happen if people from the affected domains work together on pushing such a project forward.)

Anyway, it's not my call to make; the libs teams have been pinged and we'll see what they say.

[jonathanpwang]

^Understood. Will just add that if there is a question about whether openvm makes the cut for std, then we'd like to know what is considered enough std support -- we do have the ability to add more support since the execution environment is virtual (e.g., things like time). But for example we would not consider adding full os support.