Serenity async proposal (discussion)

[axic]

serenity asynchronous execution

(Some of these details are based on asking @vbuterin questions and my interpretation of the answers. I'm not 100% familiar with the entire Serenity specification.)

Based on my limited understanding, asynchronous communication on Serenity shards operate in the following manner:

1. a transaction is created on the caller shard. It includes the destination shard number, both the caller account address and the recipient account address and some arbitrary data to be transmitted.
2. this transaction is inserted (or a reference of it) in the destination shard, is executed and an output transaction is created
3. the output transaction is inserted (or a reference of it) in the caller shard and execution of the original contract takes place

Note, cross-shard data exchange takes place between 1-2 and 2-3. This may take non-insignificant amount of time.

relevant primitives on the execution engine

A contract may issue multiple async calls in a single execution. Issuing these calls will not stop execution.

It is not clear whether it is possible or would be beneficial at all to batch incoming async response and whether that batching should be separated by shard origins.

execution dependency on async calls

Another question is whether the given account should be in a locked state until responses to its async calls are available.

Without aiming to have a complete list, here are some potential avenues to take:

1. hide all this complexity from contracts and expose a synchronous interface. This means it won't be possible to execute the contract by any other means before the receipt from the other shard arrives. The main drawback here is the contract is blocked potentially for a long amount of time. Potentially in this case contracts cannot batch multiple async calls and their execution would stop after the first one.

2. only expose a very low-level async interface and let contract developers to decide whether the contract should be in a locked state after a certain async call or not. By this I mean the contract can just set a locked flag in its regular storage - no need for any additional specific feature provided by the VM.

ewasm background

The current ewasm design is only considering synchronous execution within a single address space. It also makes heavy use of wasm's imports and exports to do effective FFI.

Methods for retrieving external data exists in the form of calldata and returndata. Additionally current contract's code and any contract's code can be retrieved. All these in practice access a buffer and it may make more sense to abstract buffers away to reduce the number of methods listed here. See #181 for more detail.

Execution starts with the main() function which receives no arguments.

async on ewasm

As we can see in order to accomplish this we need to introduce two features:

• issuing async calls;
• receiving async answers.

issuing calls

The simplest approach for issuing calls is to follow what ewasm does with synch calls. It has a call(gasLimit, address, value, data) -> result_code method exposed to contracts.

Proposal: callShard(gasLimit, shard_number, address, data)

receiving answers

The simplest way is to expose a new exported method, for example fromShard(shard_number, sender, data)

some questions

1. should we keep main() or have two more consistently named exports: fromRemoteShard() and fromCurrentShard()
2. should there be a distinction between the two invocations of the contract
3. should there be a transaction identifier returned by callShard and the same returned by fromShard so that the contract can account for the transaction if it wishes so
4. how to handle failed async calls? Include a result flag in fromShard?
5. is there a way to specify gas limit on the destination shard? Who pays the execution cost on the destination? Is the cost taken from the origin shard?

async on ewasm v2

If execution speed (aka the overhead of imported methods in wasm) is not an object, one of the more cleaner approaches would be the following.

Define a request object. It has a request kind/flags (read-only or read-write), origin shard, origin address, destination shard, destination address and a data buffer.

Define a response object. It has a flag for status, origin shard, origin address and a data buffer. It also has a reference to the request object.

Execution of a contract starts with main(response) which has a reference to the response object. The status field would indicate whether this is an external user-submitted transaction or a successful or failing async response.

To make calls two primitives exists: callSync(request) -> response and callAsync(request).

Alternative proposal: call(request) and callResult(request) -> response.

Note, this design may resemble Primea as it is a logical step in abstractions. This design also strongly depends on the success of "contract-linking" or code-merklization which would eliminate the need for the workaround of delegatecall to achieve libraries and code-deduplication.

[cdetrio]

Execution of a contract starts with main(response) which has a reference to the response object. The status field would indicate whether this is an external user-submitted transaction or a successful or failing async response. To make calls two primitives exists: callSync(request) -> response and callAsync(request).

What function on the callee is invoked when the caller does a callSync(request)? It can't be main if main is expecting a response object. Unless main looks like main(requestOrResponse) and is supposed to have handlers for both types.

More questions: does each contract have its own logic for tracking pending requests, and fulfilled requests/response objects? Or does the sharding protocol ensure guaranteed once in-order delivery of requests and responses? If not, then how are response objects validated (to prevent spoofing)? and so on.

[cdetrio]

Also note that async calls to another contract on the same shard are another use case. The difference between same-shard-async and cross-shard-async is that with same-shard-async, the contract module memory can persist, and multiple requests/responses happen in the same transaction.

With cross-shard-async, the module memory doesn't persist (only contract storage). Each request is one transaction, the response is another transaction, and so on.

[axic]

Unless main looks like main(requestOrResponse) and is supposed to have handlers for both types.

I was actually thinking about this and somehow concluded the above, with reference to the request is good enough. Probably it isn't. A better option is on_request(request) and on_response(response) for entry points.

More questions: does each contract have its own logic for tracking pending requests, and fulfilled requests/response objects?

Based on the discussion that is my impression.

Or does the sharding protocol ensure guaranteed once in-order delivery of requests and responses?

Based on the discussions it seems to me no such guarantees are provided.

Also note that async calls to another contract on the same shard are another use case.

The proposal to have call(request) and callResult(request) -> response is along the same line. I am not sure we should allow async calls on the same shard if it makes the client code more complicated.

[s1na]

I was trying to find some information about the questions you've posed, here's what I've found from here:

Identifier

should there be a transaction identifier returned by callShard and the same returned by fromShard so that the contract can account for the transaction if it wishes so

According to this, receipts have a unique id, it should be possible to return it to caller:

On shard A, destroy 5 ETH, creating a receipt...containing (i) the destination shard, (ii) the destination address, (iii) the value (5 ETH), (iv) a unique ID.

Request guarantee

Or does the sharding protocol ensure guaranteed once in-order delivery of requests and responses

I think the only guarantee might be that a "receipt" has been "spent" (not necessarily in order):

To prevent double-spends, we need to keep track in storage which receipts have already been claimed. To make this efficient, receipts need to be assigned sequential IDs. Specifically, inside each source shard, we store a next-sequence-number for each destination shard, and when a new receipt is created with source shard A and destination shard B, its sequence number is the next-sequence-number for shard B in shard A (this next-sequence-number gets incremented so it does not get reused)

Executing tx on destination shard

Who pays the execution cost on the destination? Is the cost taken from the origin shard?

I remember reading about a gas mechanism for this somewhere, but couldn't find it now. This only specifies a contract that handles receipts, but I didn't understand who calls this contract on the destination shard.

CROSS_SHARD_MESSAGE_RECEIVER: accepts as argument a CrossShardReceipt, a source_shard and a Merkle branch. Checks that the Merkle branch is valid and is rooted in a hash that the shard knows is legitimate for the source_shard, and checks that self.current_used_indices[source_shard][receipt.index] == 0. If the slot is too old, requires additional proofs to check that the proof was not already spent (see Cross-shard receipt and hibernation/waking anti-double-spending 6 foe details). If checks pass, then executes the call specified; if init_data is nonempty and the target does not exist, instantiates it with the given code and storage.

---

About the request and response:

One simplification would be to only have request and no L1 concept of response. A contract on shard A would do a callAsync(request), destination shard would take the receipt and call target somehow. The callee could in turn send a new request to caller (possibly containing request id in its data buffer). This means however that caller would have no idea what happened to the call and shouldn't rely on a response. Is this something that L2 can remedy maybe?

Asking because I feel many more things can go wrong in a cross-shard call, which takes a few blocks, and it seems complicated to prepare a response for all of the cases.

One option for contracts that absolutely would need to know the result (or need to to atomic stuff) might be to use yanking.

---

To make calls two primitives exists: callSync(request) -> response and callAsync(request)
A better option is on_request(request) and on_response(response) for entry points.

Without response in the picture, one option is to have sync calls arrive at the "main() -> (result i32)" method of callee, and async calls (from any shard) arrive at a special method on_request(request) -> void