InvariantFree trait

[HeroicKatora]

This trait indicates a type has no internal validity requirements. Even when two types are layout compatible it is not generically safe to cast between them. Types may define additional invariants of its attributes and even unsafely depend on them, such as requiring all its constructors to perform some form of global initialization.

This is in many regards a structural marker trait. A type could safely implement/derive the trait iff all of its attributes implement the trait as well, similar to the Copy trait. This is because an incorrect impl could only be written by the same crate that tries to rely on its invariants, a pure logic bug in that crate, and no other safe code is affected. Adding this trait in core would thus present a unique solution that is currently not possible to implement in a pure user-crate.

Prior art

The typic crate has a Transparent trait, that needs to be manually and unsafely implemented for types with private fields. It is the exact additional requirement that differentiates its transmute_safe casts from transmute_sound casts. However, this implementation currently has a bug as the automatic trait derive does not assert the structural condition, that all fields implement the trait as well.

In the zerocopy and bytemuck crates this trait is implicitely part of the FromBytes and Pod trait respectively.

Future directions

Similar to the Copy trait there could be an unsafe impl Transparent in the future, that would allow a type to implement the trait despite some attribute or structurally contained type not doing so. This served a very similar purpose than discussed in rust #25053 for UnsafellCell, MaybeUninit, etc. and other types where it is not possible in safe code to access the attribute of the contained type.

Another possible use would be dynamic version where the valid instance is inspected to determine if its safety invariants are met. How that interface would look could be discussed or prototyped in one of the bytecast crates, the advantages of core implementation do not apply.

[rylev]

One interesting point here is how this trait interacts with others. For instance zeroable - should the zeroable type assume Transparent? Is it worth it to know that a type has a state where it is all zeros, but that an invariant might be violated if you just treat zeros as that type?

[Lokathor]

I don't quite follow what this trait is doing, but if it's anything at all other than "this struct is repr(transparent)" then it needs a different name.

[rylev]

That’s fair. The point of this trait is “this type has no internal invariants so it’s fine to initialize one in any state.”

[RustyYato]

(Just to throw more stuff into the bikeshed, maybe Trivial/TrivialInit, because it has to invariants it is trivial to construct)

[Lokathor]

In bytemuck, @thomcc decided to call it TransparentWrapper if you can safely apply or remove the wrapper at any time.

[rylev]

Wrapper makes it sounds like this is always a new type which is not necessarily the case.
How about InvariantFree?

[casey]

I'm curious about the relationship of Transparent with FromBytes. When I initially saw the FromBytes trait, I thought that perhaps FromBytes additionally guaranteed what I currently understand Transparent to guarantee.

Correct me if I'm wrong, but if Transparent were also added, alongside FromBytes, than the relationship would be like this:

It is safe to transmute an arbitrary byte sequence of the correct size and alignment to a value of type T: FromBytes. However, it it not necessarily sound to do anything with the resulting value, like call safe methods on it, since those safe methods may be relying on invariants, and include unsafe code that relies on those invariants, which could trigger UB if the bit pattern happens to violate those invariants. To safely use the value, you would have to verify that those invariants held.

However, if T: Transparent, then it is safe to cast an arbitrary byte string to an instance of T, and then do whatever you want with it. You might get a useless/bad value, but you won't trigger UB, even if you do things like call methods on it.

Is this an accurate summary? If so, I think this would be very useful. I'm writing a serialization/deserialization library, and for some types I'd like to assert something like: "Values of this type can be created by casting from a byte sequence of the right length and alignment, however type-specific invariants must be checked before the resulting value can be used safely." but also for some other types "Values of this type can be created by casting from a byte sequence, and then no further checks are required to guarantee safety."

If Transparent existed, then it would be clear that the former would what T: FromBytes provided, and T: Transparent would be what the latter provided.

As for the name, I like InvariantFree.

[rylev]

@casey I think your definition of InvariantFree (nee Transparent) is exactly what I had in mind. We're really trying our best to have traits that only represent 1 additional guarantee over some other trait.

The only think I'm not sure of is if FromBytes should require InvariantFree. Is it useful to have a FromBytes where the type can be constructed from correctly sized/aligned byte arrays but cannot necessarily be used unless invariants are checked? Ultimately, there should be some trait that represents types that can be constructed and used from sufficiently sized/aligned byte arrays. The question is, is FromBytes that trait and if so, do we have another trait that represents safe to construct but not safe to use types?

[rylev]

I'm also not sure if we should make a distinction between types that require their invariants to be checked to used safely vs types that require invariants to be checked to be used correctly. This is a subtle distinction, but I can imagine safe could that requires a caller to verify invariants in order for certain operations to be used correctly even though if they are used incorrectly it does not lead to unsafe code (but perhaps to wrong results or panics).

[casey]

@casey I think your definition of InvariantFree (nee Transparent) is exactly what I had in mind. We're really trying our best to have traits that only represent 1 additional guarantee over some other trait.

If that's the case, than I think that FromBytes shouldn't require
InvariantFree, since they seem like distinct guarantees.

The only think I'm not sure of is if FromBytes should require InvariantFree.

If FromBytes requires InvariantFree, than I'm not sure what InvariantFree
means on its own. The guarantee that T: InvariantFree is making is that
arbitrary byte sequences of the correct length are valid values of type T.
However, this guarantee only seems useful if there is some way of bypassing the
visibility protections on the type's fields. If there is no way of doing that
(e.g. no FromBytes that lets external users directly construct a value from
bytes), then I'm not sure if that guarantee is useful, since it couldn't be
violated, even if it wasn't true.

Is it useful to have a FromBytes where the type can be constructed from correctly sized/aligned byte arrays but cannot necessarily be used unless invariants are checked?

In my own codebase, I have a lot of unsafe casts that I believe are always
safe, but where it is not necessarily safe to use the resulting value. (I can
provide some examples, if that's useful.) So safe construction but not safe use
would be useful.

As a side note, even if those casts are safe, the functions that perform them
should still be marked as unsafe, e.g. unsafe fn safe_cast<T, U>(T) -> U,
because it would be rather dangerous to have a safe function return a value
that cannot be used safely. fn safe_cast<T, U>(T) -> MaybeUninit<U> would
also make sense.

Ultimately, there should be some trait that represents types that can be constructed and used from sufficiently sized/aligned byte arrays. The question is, is FromBytes that trait and if so, do we have another trait that represents safe to construct but not safe to use types?

It's probably a good idea to separate the naming of those traits from whether
or not mem-markers should provide them. If FromBytes produces a value that
isn't safe to use, then it should probably have a scarier name, since otherwise
it sounds like it's safe.

I'm also not sure if we should make a distinction between types that require their invariants to be checked to used safely vs types that require invariants to be checked to be used correctly. This is a subtle distinction, but I can imagine safe could that requires a caller to verify invariants in order for certain operations to be used correctly even though if they are used incorrectly it does not lead to unsafe code (but perhaps to wrong results or panics).

I personally think this distinction is useful, because there is such a huge
difference between a value that is unsafe and value that is incorrect. If a
value is unsafe, and doing things with it might invoke UB, and then that UB
means that all bets and guarantees are off, that seems like a big difference
between a value that might not be correct, and thus might cause conventional
bugs if used.