Add a native function to copy a slice from a vector

[gfusee]

Abstract

Manipulating data vectors is common in Move packages. A fundamental operation is the ability to retrieve a slice of a vector in the range of [start; end[.

A particular use case is when handling a vector of u8 (a.k.a bytes). Currently, a developer must copy the bytes one by one from the original vector to a new one to create the slice. This is far from efficient, as it involves performing a number of read, copy, and write operations equal to the length of the slice range.

Solution proposed

To address this, I propose implementing a new native function, public native fun vector::copy_slice<T: copy>(v: &vector<T>, start: u64, end: u64): vector<T>. Constraining T to have the copy ability is mandatory to ensure that no Move rule is violated when copying the data into a slice.

As a native function, it would take advantage of the Rust runtime to read the container, copy the values once, and return them as a vector<T>.

If you agree with this idea, I can work on it and submit a PR 🙏

(cc @damirka)

[tnowacki]

I don't see the need to make the function native.
Though it is an interesting one to discuss adding to the module std::vector. We had discussed something similar when adding take and skip though we weren't sure how necessary or common patterns like this are.

Assuming you do not want it to abort, you can implement this as.

public fun copy_slice<T: copy>(v: &vector<T>, start: u64, end: u64): vector<T> {
    let end = end.max(v.length());
    let mut result = vector[];
    start.range_do!(end, |i| result.push_back(v[i]));
    result
}

[gfusee]

The issue is that not making this function native makes slice manipulation inefficient.

Let's suppose the range has a length of 1000. It means we have to call:

• the getter v[i] 1000 times
• copy v[i] (implicitly) 1000 times
• push_back 1000 times

Each of these actions calls the Rust runtime and creates new heap allocations or changes.

It also implies charging unfair gas fees because, for example, the gas cost for push_back includes a base_cost fee that doesn’t take into account the size of the value:
base_cost + legacy_unit_cost * max(1, size_of(val)).
On the current mainnet config, base_cost is 52 and legacy_unit_cost is 2, which means we have to pay 52 * 1000 = 52000.
So, pushing 1000 values on a vector<u8> costs (52 + 2 * 16) * 1000 = 84000. 61.9% of the gas consumed is not due to the size of the value, but to an arbitrary fee. The same reasoning applies to the native function borrow called by v[i].

For all these reasons, doing that many operations will quickly deplete all the available gas for the transaction.

By marking it native, we can take advantage of the Rust runtime, drastically reducing the number of actions required and making them more efficient in terms of memory management. For instance, the Container enum contains all the information needed to perform super-efficient slicing. It’s even better for Container cases where the underlying Vec<T> type T is Copy (Container::VecU8, Container::VecU64, Container::VecAddress, ...).

There are countless use cases for slicing, but a particular one is the vector<u8> type, where efficient slicing would have immediate positive effects. This operation is very common when doing low-level work, and as an example, we can think of almost all the functions in the sui::bcs module, which would benefit from this (especially the peel_num! macro function).

Outside of the Sui framework, manipulating bytes can also be necessary when receiving data from outside the chain (through oracles, Nautilus-based setups, ABI-encoded Solidity data using Axelar/Wormhole, ...).

[tnowacki]

(I will refer to "faster" and "efficiency" here both in terms of speed and gas cost)

Completely agree that certain things can be made faster by use of native functions, but this in general has not been a motivating factor for what becomes native.
While not a hard rule, the two main use cases for native have been expressibility within Move, and using libraries defined outside of Move. The latter has been primarily for crypto/math functions and we can ignore those for now.
For expressibility, this is the reason why vector::swap is native and vector::reverse is not. Certainly, reverse could be made faster by going native, but it is not a primary concern.

There are likely some missing functions from std::vector in terms of expressibility, but at this time, we do not think that this is one of them.

---

A few other things to note (in no particular order)

• Code you write is likely never calling the native function for push_back and instead should be calling Bytecode::VecPushBack (I will double check this since otherwise there is a compiler bug)
• The gas costs for natives have not been carefully optimized, and more designed right now to prevent exploitation. We should do a pass at some point for "fairness"
• The VM is being rewritten. We will likely reanalyze gas over that new interpreter, so any gas changes might be a while.