Configurable and composable storage

[ironcev]

Rendered

This RFC introduces a concept of a Storage trait, as well as the StorageBox and StorageEncodedBox structs. Those types are cornerstones upon which we build a fully flexible, feature rich, and robust access to storage. These concepts simplify defining dynamic storage types, like e.g., StorageVec. They also provide a dedicated way of configuring (in the storage declaration) and initializing (in the code) arbitrary storage types composed of existing storage types.

Additionally, the RFC provides API design guidelines for storage type's APIs. Those guidelines ensure that various aspects of a storage access like, e.g., accessing uninitialized storage, are alway treated in the same way, across various storage types.

Closes #34.

[IGI-111]

We discussed changing the naming in the API previously here: FuelLabs/sway#4740

I think we should go with the opposite of what's said in this paragraph.

Most of the time, you want an operation to return Option. I'd much rather we standardize on read+read_unchecked than try_read + read. Reverts should be the exception not the rule and it should be clear when an operation potentially reverts.

[ironcev]

TL; DR

The proposal (for the next discussion round 😄) is:

• fn <operation>() -> Option<T>: None in case of uninitialized storage or other, semantic, error.
• fn <operation>_or_revert() -> T
• fn <operation>_<special_behavior>() -> Option<T>: None in case of uninitialized storage or other, semantic, error.
• fn <operation>_<special_behavior>_or_revert() -> T
• fn <operation>() -> T: operation never fails.
• fn <operation>_<special_behavior>() -> T: operation with special behavior never fails.

Looking at the storage types we have, there will never be an explosion of those variants per operation, and we still have a clear guidelines that capture the overall complexity (explained below).

Detailed Analysis

After spending a good portion of the last week walking in circles and pondering how to capture the complexity of the storage access semantics into consistent API guidelines, here I am again, doing the same 😄

I am completely fine with promoting the Option returning operations to be the default ones.

But we still need to capture the overall matrix of possible combinations. Since FuelLabs/sway/#4740 is not going into details there, let's do it here.

Before starting, just to emphasise the difference, the term "unchecked" in the #4740 and in the RFC paragraph, means two different things. In the #4740 it is a synonim for unwrap while in the RFC it represents a "special behavior". In the concrete example, running the StorageVec::get() without checking the vector length and out-of-bounds access.

To the matrix of possible combinations and what we want to capture in the API.

Every operation can potentially:

• fail if the storage is not initialized. E.g., StorageBox::read() on uninitialized StorageBox.
• fail because of a semantically wrong usage. E.g., out of bounds StorageVec::get() on initialized StorageVec.
• have one or more "special behaviors" that affect what the operation do. E.g., StorageVec::get() without lenght checks. E.g., StorageVec::pop() with deep clear.
  • we expect a small number of special behaviors per operation, very often none or mostly one.
  • special behaviors might require additional trait constraints. E.g. DeepClearStorage.
  • special behavior operations can also fail because of the first two points.
• have none of the above; neither it can fail nor it has any special behaviors. E.g., StorageMap::get().

The guidelines should reflect the above matrix.

What RFC proposes, to simplify the matrix and the API, and assuming that we are not losing any valuable information there, is to treat the access to the uninitialized storage as a semantically wrong usage and not as a special technical error.

Means, every storage type assigns a meaningful default semantics to the uninitialized state. E.g, an uninitialized StorageVec behaves the same as an empty StorageVec. For all the storage types we currently support, this is possible in an intuitive way.

It also has a benefit of just creating an instance with new() without the need for initialization. StorageVec::new() behaves the same as StorageVec::init([]) but does not produce any storage writes until used. For storage types I've played with and the sample StoragePair having this meaningful default behavior was always possible.

If we do not join those two points and consistently want to distinguish between the failing because of the storage being uninitialized and because of a semantically wrong usage, we will necessarily end up in the cumbersome API like, e.g., StorageVec::get() returning Option<Option<T>> where the first option means failure on uninitialized and the second out-of-bounds. I doubt that developers will ever want to distinguish those two cases.

One more thing to consider in the guidelines is:

it should be clear when an operation potentially reverts

How to achieve this via naming convention only? (We are talking here about the computational effects. We had this discussion internally triggered by the #[storage] attributes not being a part of the function type.)

One possibility would be:

• <operation>() -> Option<T>: does not revert.
• <operation>_<unchecked or some better name>() -> T: potentially reverts.
• <operation>() -> T: does not revert. E.g., StorageMap::get() -> T.

And all this should also be aplicable to "special behavior" variants.

Using the term "unchecked" as a suffix I see as misleading. Because, in the case of, e.g., StorageBox::read_unchecked() it actually does check and it will revert if that check fails.

"unchecked" as a word implies that we are actually skipping checks and who knows what can happen. Developers responsibility. E.g., in the RFC, the StorageVec::get_unchecked() cannot fail because it literally does not check the bounds and returns the calculated element. Accessing that element later on can fail, but the get_unchecked() itself cannot.

I am arguing here to adopt some other terminology for the <operation>_<give me the result or revert if it is not there>(), and not not to have that variant.

Perhaps <operation>_or_revert()?

To wrap it up, the proposal (for the next discussion round 😄) is:

• fn <operation>() -> Option<T>: None in case of uninitialized storage or other, semantic, error.
• fn <operation>_or_revert() -> T
• fn <operation>_<special_behavior>() -> Option<T>: None in case of uninitialized storage or other, semantic, error.
• fn <operation>_<special_behavior>_or_revert() -> T
• fn <operation>() -> T: operation never fails.
• fn <operation>_<special_behavior>() -> T: operation with special behavior never fails.

Looking at the storage types we have, there will never be an explosion of those variants per operation, and we still have a clear guidelines that capture the overall complexity.

[ironcev]

A question that immediately comes to my mind is why _or_revert()/unchecked() at all, when we can simply call unwrap()?

The <operation>() <-> try_<operation>() pairs are there from the beginning if that's a valid reason. <operation>().unwrap() results in a potentially much bigger code then <operation>_or_revert() (see FuelLabs/sway/#5982), but this should be handled in optimizations.

Also, other APIs, aside from the storage API, does not have such pairs. E.g. Vec has only get() -> Option<T>.

So why to have _or_revert()/unchecked() so prominently in the storage API?

[IGI-111]

I think StorageRef is an unnecessary complication.

I don't see what this adds over Option<StorageKey> which would already be supported. If anything it is worse because it reintroduces a potential implicit null.

Unless there's a compelling reason to have a special type, I say we should remove this and recommend storing StorageKeys and Option<StorageKey>s for references to other parts of the storage.

[ironcev]

Removing the enum and the Null and using Option instead I like a lot!

Actually, the first version of the StorageRef was a struct type-safely wrapping the StorageKey but then I ended up thinking how to support not having a reference and introduced Null motivated by the similar programming language concept. The result was putting the burden of checking for Null when dereferencing always, even in the situations where the reference will always be there. Using Option<StorageRef> where the StorageRef is a struct is definitely the right way!

To the question of having the dedicated StorageRef, or using the general StorageKey directly, the benefit of having the struct StorageRef<TStorage> is there, because of these two reasons:

1. Type-safety.
2. Having easy-to-use, zero-(additional)-cost abstraction that clearly communicates the intent and provides a standardized usage pattern.

Ad 1) StorageRef<TStorage> referrs to a storage of a particular type and that is type-checked by the compiler. It also clearly communicates that it referrs to a storage of a particular type. Using StorageKey would be more like having a raw pointer into the storage.

Ad 2) Using the naked StorageKey will require from developers to implement their own deref equivalents every time they use it for referencing. Those will be cumbersome and error-prone, because they will need to use the proper way to create the proper storage type etc. In the worse case, we will end up having that code in contracts, and in the best case, having it packed in various libraries. Having the StorageRef provided by the standard library gives a simple type that clearly communicates the intent and gives all convenience in a standardized and zero-cost way.

[IGI-111]

The mechanism is sound, but we need better naming than "config". It should be clearer that StorageConfig represents a single address initializer for consecutive slots and that Config is a whole, potentially composed, initializer.

Maybe StorageInit and StorageAddressInit?

[ironcev]

Thanks for the proposal! I am not happy with the term Config either. While the Value as the name of the associated type is intuitive, Config does not carry such clarity.

Here we need to be cautios of the overall "initialization" vs "configuration" confusion. Since configurables and storage are actually configured in code and that configuration can be changed at deployment time, the term "configuration" is consistently used everywhere in the RFC. Sticking to that term will help to remove existing confusion, where the choice of the syntax and wording implies "initialization" and "assignment" to something that is rather a "configuration".

E.g., the Storage::init constructor on the other side, is an "initialization" because it will create a storge type instance at the runtime and actually stores the provided values into the storage.

To the origin of the current, non-optimal name Config, which I tried to change (my thinking below). Since both the associated type Config and the StorageConfig are used only by the compiler during the configuration, I've kept the name Config although I don't like it.

Alternative names I've played with for the Config associated type were:

• ValueLayout: because it provides the information on how the value (described by the Value type) is layed out in the storage. The reason I didn't choose it is, that storage types can store additional information as well, e.g., StorageVec its length, and not only what is in the Value.
• StorageLayout: it clashed with the StorageLayout enum.
• StorageContent: because it describes which content is stored in the storage and where exactly.

Using "Init" in the name, as explained about, can lead back to the current confusion.

Perhaps SlotsConfig because it provides the information on how to configure the slots, or even better SlotsContent?

Additional proposals are welcome 😄

[ironcev]

Also, regardless of the terminology we chose at the end, I like the proposed formulation (but to change "initializing" to "configuring"):

StorageConfig represents a single address initializer for consecutive slots and that Config is a whole, potentially composed, initializer

Hmmm, perhaps:

Storage(Address)Config/Content represents a configuration for storing a single value/content in consecutive slots starting at the address specified by the storage_key. The SlotsConfig/Content is a whole, potentially composed, configuration, for storing the entire value contained in the Storage.

Whatever the final terminology will be, it should sound well in that sentence 😄

[IGI-111]

Capitalized STD is confusing because the standard library is already made up of std and core, either use those specifically or just say "the standard library".

[IGI-111]

Direct programmatic manipulation of storage should not be encouraged but it should not be this discouraged either. There are cases where it's a better more straightforward pattern to use to implement what you want.

I do think having it in a separate module is fine, but I don't like the implication that it is "internal", it's merly raw or low level. But it's perfectly valid for the end user and not specific to storage type implementors.

[ironcev]

Agree!

Encouraging and/or discouraging usage is a separate topic, similar to the one of using raw pointers in contract code. It is to be reflected in the way how we convey and document the library/language features, patterns, etc. But in both cases, even discouraging the usage does not justify the name "internal".

The current module name is storage_api which we can keep. But, I am more for raw_api or some other classifier which will better convay that this is the low-level API, rarely needed, if at all, in the contract code.

@bitzoic Do you have a view or proposal here?

[IGI-111]

Maybe we should have some discussion of the performance implications of this change. Can storage get slower as a result of this, if employed correctly?

[ironcev]

I say - no 😄 (to the negative implications, not to the discussion 😄)

Or at least I don't see any negative impact on the performance, compared to the approach we have now. On the contrary:

• Performance was in focus and considered carefully. It resulted, e.g, in the StorageLayout enum being introduced.
• Just using the StorageLayout information consistently will help Storage implementors to provide implementations that optimally pack and store data across the whole hierarchy of the stored data.
• Having the convention for "special behavior" methods should encourage providing optimized versions of methods where high-performance is needed and safety guaranteed by surrounding code.

But it can be that I am missing something, so yes we can surely discuss it further. And of course reviewing the proposal from the performance standpoint is highly encouraged.

[IGI-111]

There are important reasons (for standards, notably) that we cannot be explicitly bound to read and write and must have read only/read write semantics.

Maybe we can just have the smallest change:

#[storage(read)]
#[storage(read_write)]

[IGI-111]

If we go with StorageInit instead of Config, maybe this can be internal_make_initializer

[ironcev]

I like the verb make! I didn't like get and was thinking about alternatives, but didn't like them, e.g. create, either. They all sounded very "runtimeysh". make somehow does not have that apeal. Here is your self key and the value to store, make me the <config (good name still to find 😄)>.

As mentioned, initializer can mislead and in the and, the final name behind make will depend on how we name the Config associated type.

[IGI-111]

The name of "self key" is an artifact of our thought process and it's littered all over this proposal, but I think we should try to come up with a better one since it's no longer involved with self as a keyword.

Maybe "field key"? Since it's the StorageKey for a given storage field?

[ironcev]

Well, self_key is the best name 😄 IMO, of course.

It very precisely communicates what it is, and that's why it is introduced and consistently used in the RFC:

An important term to establish will be the self key. Self key is the StorageKey at which an instance of a storage type is stored.

Why I see it having properties of a perfectly good name:

• In the implementations of Storage types, the term self_key immediately communicates that "this is my storage key, the storage key at which I am stored".
• The public self_key() method is equally clear. It gives a clear answer to the question "what is your key?". (We need this method for, e.g., referencing or any other scenario where the knowledge of the slot location of the storage type instance is needed.)
• In the documentation and communication, being reflexive it immediately points to the storage instance we are talking about.

Any other word that does not imply reflexive relationship (like "self", or "me") will be to broad and possibly applicable to several instances and the question "whose, of what" will not be immediately incorporated in the word.

E.g., having the field_key everywhere rises the question "which field? whose field?". The sentence "Field key is the StorageKey at which the instance of a storage type is stored" calls for explaining why the word "field". We have to keep in mind that storage type instances can be used independent of the storage which makes the term "field" even less understandable and hard to justify.

Also, some storage types could have clashing terminology with the word we choose. E.g., if we chose "element key", it will clash with what an element in the StorageVec is.

That's why I am for the name that implies reflexive relationship and see "self" as the perfect candidate. It also corresponds to what "self" is in Sway types. Something that points to, or denotes, myself.

[bitzoic]

On first look, I am unsure how I feel about including StorageBox in the storage definition from a user perspective. It makes sense however with the current storage implementation the StorageKey is behind the scenes and makes it much more readable:

storage {
     box_1: StorageBox<u64> := 0,
}

vs

storage {
     box_1: u64 = 0, // This is actually a StorageKey<u64>
}

It is unclear for me whether this is still achievable on the compiler side.

As AbiEncode is implemented for primitive types such as u64, couldn't StorageEncodedBox be used? How do we know we are using using "the correct one"? A lot of the confusion in storage that we have seen from external developers is around having separate types for storage i.e. StorageVec vs Vec. Requiring that developers need to make the choice between StorageBox and StorageEncodedBox could further that confusion.

I am also wondering what the difference is here between StorageVec and Vec:

vec_of_encoded_val_1: StorageVec<StorageEncodedBox<Vec<bool>> := [Vec::from([true, false, true]), Vec::default(), Vec::from([true])],

As Vec is a heap type and is supported, wouldn't this obsolete StorageVec? In what case would I use one over the other? How would we ensure that developers know which to use?

I am very much so liking the use of write_deep_clear and it's associated functions as well as StorageLayout

[IGI-111]

@bitzoic A big part of this change is trying to separate and give a proper interface between the structure of what is being stored and the storage method itself.

This is why we can't stay with implicit u64 fields: you should tell the compiler how you want things stored, and indeed you should actually have to make that decision. Encoding has benefits but it also has a cost you may not want to pay, etc.

Being explicit about the storage types also has the nice benefit that storage.a is actually of the type of field a in storage, which means you can easily mentally model what's happening with no magic.

It also means that we do not need to be opinionated as to how people store things, and they can build their own storage data structures as libraries (as is already being done by some users).

[ironcev]

@bitzoic Why we insist on StorageBox<u64> is already explained and the clarity of the intention is the main reason. Let me just add another one reason, although being just a consequence of the composability, why inferring StorageBox usage from the compiler in a general case would be impossible.

Currently, whatever the type in the storage field declaration is, it is wrapped in the StorageKey. With the composable storage, that's not the case. Consider, e.g., StorageMap<str[3], StorageBox<u64>>. If we allow for StorageMap<str[3], u64> and expect the compiler to infer StorageBoxes, how can the compiler know, in a general case, which type arguments to wrap within the StorageBox and which not.

The questions regarding the choices between the types are great! I expect them to be well addressed in the documentation. Actually, discussing them was also a part of the initial RFC, but I've removed them to shorten the overall length of already lengthy RFC. But I think now I'll put them back, at least a short explanation.

As AbiEncode is implemented for primitive types such as u64, couldn't StorageEncodedBox be used? How do we know we are using using "the correct one"?

Currently, the negative impl in the StorageEncodedBox forbids its usage with the types that can be stored without encoding. There is a TODO-DISCUSSION comment above asking if this should be a hard error or a compiler warning.

In both cases, developers will be properly informed and guided by the compiler if they tried to do StorageEncodedBox<u64>.

I am also wondering what the difference is here between StorageVec and Vec
As Vec is a heap type and is supported, wouldn't this obsolete StorageVec? In what case would I use one over the other? How would we ensure that developers know which to use?

StorageVec provides access to individual vector elements, thus alowing pushing and reading individual elements in a manner that minimizes the number of storage reads and writes.

StorageEncodedBox<Vec> on the other side always reads and writes back the entire vector. This is very likely not what developers want and we will explain the difference.

The intended usage of the StorageEncodedBox is with custom dynamic Sway types that thus must be encoded to be stored but are expected to be read entirely, like the Struct used in the example that contains a Vec and a String.

So, I expect having something like StorageVec<StorageEncodedBox<MyTypeThatNeedsEncoding>> to be seen.

The interesting consequence of this is that the StorageString will be obsolete or just become a type alias for the StorageEncodedBox<String>. Because in this case we want to store and read the entire string and not having each character be stored and accessable separately within the storage.

I'll address this in the RFC. We will definitely provide sufficient guidelines and examples in the documentation so that developers know which storage type to use.

[SwayStar123]

@ironcev Is there any situation in which i can use StorageBox and would want to use StorageEncodedBox instead?

[esdrubal]

The RFC looks good to me and is a noticeable improvement on what we have now.

I noticed that StorageBox<T> requires T to implement Serializable and StorageEncodedBox<T> requires T to implement AbiEncode + AbiDecode.
The RFC also mentions that StorageBox is more gas efficient than StorageEncodedBox, the question I have Is why is it more efficient? How would Serializable trait work? We cannot copy the contiguous memory if we assume the memory representation of types can change in the future.

[ironcev]

Is there any situation in which i can use StorageBox and would want to use StorageEncodedBox instead?

@SwayStar123 I am not aware of any. In the proposal, this line:

impl<T> !Storage for StorageEncodedBox<T> where T: Serializable { }

forbids using StorageEncodedBox if the type can be stored using StorageBox.

However, I've left the discussion question asking the same as you, is there a valid reason to allow that and treat it as a warning rather then forbidding it:

// TODO-DISCUSSION: Shell we forbid encoded-boxing serializable types and thus force
//                  them to be boxed in `StorageBox` or should this only be a compiler warning?
//                  Essentially, if a type is `Serializable` encoding it unnecessarily
//                  is a huge waste of computational and storage resources.

Note that we can always start as proposed, having a compiler error if a serializable type and thus StorageBoxable type is attempted to be stored in EncodedStorageBox and later on remove the restriction and replace it with a warning, without having a breaking change.

[ironcev]

The RFC also mentions that StorageBox is more gas efficient than StorageEncodedBox, the question I have Is why is it more efficient? How would Serializable trait work? We cannot copy the contiguous memory if we assume the memory representation of types can change in the future.

@esdrubal Currently, this is exactly what we do when storing types into storage, copying memory representation as is. At compile time storage configuration uses the Sway memory layout information to calculate the slots. The Serializable marker trait puts the safety around copying memory, ensuring that we do not copy types that, e.g., contain pointers. The similar guard is there when we autoimpl AbiEncode.

If the memory representation changes in the future, the issue might indeed arise if we e.g. have code accessing storage that gets LDCed into a contract that used old memory layout. This is a valid concern. I see it being in the same category like some others where we traded undefined behavior for performance. Paying for decoding/encoding at every storage access would definitely be a very high cost to pay for a change of memory layout that we will potentially never introduce. Also, note that we would have a similar issue with the StorageEncodedBox if the encoding changes from v1 to vNext.

That's why I am for addressing this concern as a breaking change in existing contracts if it ever happens.

As a side note, what we might introduce in the future is the configurable memory layout equivalent to #[repr(...)] in Rust: FuelLabs/sway/#5286

In that case, we would still be copying the memory. The compiler would be aware of the #[repr] attribute and during configuration calculate the slots accordingly. The above concern would remain or be even more prominent. E.g. having the same struct but with two different #[repr]s in different contracts accessing the same storage.

CC: @IGI-111

[SwayStar123]

Is there any situation in which i can use StorageBox and would want to use StorageEncodedBox instead?

@SwayStar123 I am not aware of any. In the proposal, this line:

impl<T> !Storage for StorageEncodedBox<T> where T: Serializable { }

forbids using StorageEncodedBox if the type can be stored using StorageBox.

However, I've left the discussion question asking the same as you, is there a valid reason to allow that and treat it as a warning rather then forbidding it:

// TODO-DISCUSSION: Shell we forbid encoded-boxing serializable types and thus force
//                  them to be boxed in `StorageBox` or should this only be a compiler warning?
//                  Essentially, if a type is `Serializable` encoding it unnecessarily
//                  is a huge waste of computational and storage resources.

Note that we can always start as proposed, having a compiler error if a serializable type and thus StorageBoxable type is attempted to be stored in EncodedStorageBox and later on remove the restriction and replace it with a warning, without having a breaking change.

Im confused then, the stated reason for having explicit storagebox and storageencodedbox is that the developer should make a conscious decision to choose between them, but apparently theres no choice at all?

[ironcev]

Here are some additional remarks and concerns coming from discussion with @SwayStar123.

1. StorageBox and EncodedStorageBox. Unnecessary for the developer to have to think about this. Can't the compiler just figure out which one is necessary?

   The reasoning for explicitly using StorageBox and EncodedStorageBox are already discussed in detail here:
   Configurable and composable storage #40 (comment)
   Configurable and composable storage #40 (comment)

   I would like to additionally point out, that in general, developers will actually not need to think about the existance of two stroage boxes. In vast majority of cases, it is simply going to be the StroageBox. We will need to properly convey the mental model of our storage approach. Storage types offer structural view over the storage creating hierarchies ending up in atomic storage boxes. Those will, for the majority of cases be the StorageBox.

   If the EncodedStorageBox is needed, and a developer still habitually picks StorageBox, the compiler error will clearly explain why the encoded version is needed. Expressive diagnostics enable us to help developers learn on their own code.

   My point is, in general, there will be no real need to think much upfront about the choice between two storage boxes.

2. The fact that there are non interchangable vec and storagevec. Strongly against this design. Not only will this be hard to communicate to all devs, it's also adding complexity as compared to solidity, is it not supposed to be the opposite?

   This is a direct consequence of two design decisions in Sway as a language and our approach to storage, established with Storage v2.0.

   • First, unlike dynamic arrays in Solidity, Sway's vectors are not a language construct but rather a library type. From the compilers perspective, there is no any difference between the Vec and any other ordinary Sway struct. Vec is just a struct that internally does memory allocation.
   • Second, Sway transparently presents the nature of the contract storage. It has the explicit storage declaration to declare storage "variables" and insist on reading and writing to storage via methods that clearly convey that they might fail. The storage access is intentionally conceptually presented as IO access unlike accessing variables in memory.

   The combination of these two design choices makes StorageVec and Vec different in the API and not interchangeable. Even if we would built-in special support for Vec into compiler (not something I would advocate) and compile it differently when in memory and in storage, Vec's API as such wouldn't fit to storage API.

   The perceived complexity, which I fully agree is there, needs to be communicated through the lenses of the benefits of the Sway approach to storage. Here I like how @IGI-111 put it, "no magic" + "flexibility":

   Being explicit about the storage types also has the nice benefit that storage.a is actually of the type of field a in storage, which means you can easily mentally model what's happening with no magic.
   It also means that we do not need to be opinionated as to how people store things, and they can build their own storage data structures as libraries

3. I can imagine many devs not even knowing storagevec existing and using horribly inefficient code where they read and write the entire vecs everytime they want to update an element

   With a strong background in a managed language simple to use in an unperformant way, I also see this as a problem.

   The RFC pays a lot of attention to providing the infrastructure and the API that will be difficult to use in a false way, both when defining stroage types and using them. This is one example where, purely throught the API it is not possible to guide the developers to the right choice. On the other side, treating Vec differently on the compiler side, or forbidding storing dynamic structs is also not an option.

   The question is what to do to mitigate the potential unintended misuse.

   I see two complementary approaches here. The documentation will have StorageVec as the default narrative, as it is now. Also the StorageBox. The EncodedStorageBox<Vec> should be on the margins and explained with red flag and as potential optimization and very likely not the regular case. In addition, we've discussed standardized static analysis tool similar to Clippy for Sway. E.g., Sway Analyzer can give a warning when using encoded Vec in storage.

   But to come back to the flexibility point, storing EncodedStorageBox<Vec> can be the more performant option in certain scenarios. E.g., small vectors of u8 that need to be loaded and stored completely, etc.

[ironcev]

Im confused then, the stated reason for having explicit storagebox and storageencodedbox is that the developer should make a conscious decision to choose between them, but apparently theres no choice at all?

Fair point 😄 We've discussed the topic of choice and the need for both the StorageBox and the StorageEncodedBox in the team meeting. The discussion addresses this question and solves as well the open TODO-DISCUSSION point that was asking if we should forbid storing serializable types in the StorageEncodedBox.

Please see the Explain difference between StorageBox and the StorageEncodedBox commit for the conclusion.

CC: @SwayStar123, @esdrubal, @IGI-111

[ironcev]

Hmm, the two dead links in the collection_context RFC actually exist, both the old 2019 ones, and the newer, but the link verification still fails.

[✖] https://papl.cs.brown.edu/2020/types.html#(part._.Recursive_.Datatype_.Definitions) → Status: 0
[✖] https://papl.cs.brown.edu/2020/Type_Inference.html → Status: 0

[Nazeeh21]

By extending the convo with @ironcev from Slack I would like to propose a concept/implementation for the StorageMap concept. I along with one of the SRs from the attackathon, have put all the info regarding that in the gist here along with the example implementation. It might be a bit clumsy right now, but at least it presents the idea. I reworked a bit the original StorageKey concept and combined it with the existing StorageMap, the outcome is a single struct.

I think it should also support nested maps, but need to test it.

The API of this SecureStorageMap thing is a bit tricky to use as methods and constructors return Result, so maybe that could be somehow improved. Let me know what you think about it.