Reduce state/tipset/msg execution/replay/compute APIs in /v2

[rvagg]

Breaking this out from #13027 where I was starting to ramble a bit too much and departing from the issue at hand.

---

Given we are working on /v2, there's an opportunity to consolidate our msg execution/replay APIs.

We have StateReplay and StateCall and StateCompute today and they all do roughly the same thing but with minor variations, and the internals are becoming very messy to support the variations while keeping backward compatibility, made more complicated by the needs of the Ethereum APIs which want to do similar things.

@Stebalien and I were discussing these the other day trying to boil them down to a single method that we can overload with an options struct argument to support the variations we care about. Then we can push those options down into the core and clean things up a bit.

In #12822 I've replaced a use of StateReplay with StateCall and I'm having to introduce third argument to it just to get my new option in, but in that case I'm introducing an optional struct argument (optional so it's backward compatible as JSONRPC) so I can get the new option in.

callInternal already illustrates the problem if you look both at the number of args, and the number of branches in it:

lotus/chain/stmgr/call.go

Line 93 in 6f09aca

func (sm *StateManager) callInternal(ctx context.Context, msg *types.Message, priorMsgs []types.ChainMsg, ts *types.TipSet, stateCid cid.Cid,

- that was most recently complicated by needing a new "strategy" for Ethereum APIs.

Then, if you look at how StateReplay works, you see that it's basically the same thing as StateCall, but with a "execute everything up to this msg" strategy: https://github.com/filecoin-project/lotus/blob/6f09aca67ac53f65004751f55e6aa3c86b349153/chain/stmgr/call.go#L295-L311

StateCompute is something like a StateCall but with an batch of messages and a "apply all messages, then apply these new ones" strategy but also return the new state root.

Right now we're considering at adding a state root to InvocResult as part of solving #13022, which would make ComputeStateOutput redundant.

So, I think we could introduce something like:

StateExecute({
  BaseTipSet, // mandatory
  []ImplicitMessages, // optional, could be used to solve some of our gas estimation bugs
  []Messages, // optional
  TipSetMessageSelectionStrategy, // optional, default to "all", could also be "none", "same sender"
  FlushType, // optional, defaults "none", could be "state", or "all"
})

[rvagg]

Here's another fun factoid: it looks like all of our current uses of StateCompute in lotus use the same height argument as the tipset they are calling with:

lotus/cmd/lotus-shed/chain.go

Line 81 in 6f09aca

if _, err := api.StateCompute(ctx, startTs.Height(), nil, startTs.Key()); err != nil {

lotus/cmd/lotus-shed/chain.go

Lines 91 to 97 in 6f09aca

	ts, err := api.ChainGetTipSetByHeight(ctx, height, endTs.Key())
	if err != nil {
	return err
	}
	if _, err := api.StateCompute(ctx, height, nil, ts.Key()); err != nil {
	return err

lotus/cmd/lotus-shed/chain.go

Line 102 in 6f09aca

if _, err := api.StateCompute(ctx, endTs.Height(), nil, endTs.Key()); err != nil {

lotus/cmd/lotus-shed/miner-fees.go

Line 195 in 6f09aca

compute, err := api.StateCompute(ctx, dlEndTs.Height(), nil, dlEndTs.Key())

lotus/itests/migration_test.go

Line 403 in 6f09aca

cso, err := clientApi.StateCompute(ctx, currTs.Height(), nil, currTs.Key())

lotus/cmd/lotus-shed/mismatches.go

Line 40 in 6f09aca

st, err := api.StateCompute(ctx, execTs.Height(), nil, execTsk)

In all of the above, it's only the last one where we care about the Root return value, so the others could probably be easily replaced with a variation of StateCall or StateReplay.

lotus state compute-state is the only place where you can differ the height from the tipset, but it exists as a CLI form of the API, so that makes sense.

[rvagg]

Something else to consider here - StateCompute does lean on an execution trace cache, because the strategy involves re-executing a complete tipset (even if you're going to apply new msgs on top), it can cache old ones to make it more efficient. It only defaults to 16 because they can be large objects, but I think Glif might bump that up in production. But that could easily be part of some new internal form of this thing - "if we're doing all messages in this tipset, use a cache".

[masih]

it looks like all of our current uses of StateCompute in lotus use the same height argument as the tipset they are calling with

This is consistent with what I observed in Zondax codebase and the rationale for removing height from StateCompute implementation in #13027

[rvagg]

OK, so it just becomes the same as a full-messages StateReplay but gives you the root state CID in the return, which is nice, but it should just be in InvocResult rather than having a special ComputeStateOutput to contain it.

[Kubuxu]

StateCompute with explicit height, afaik came into existence for null-epoch and migration testing.

[Stebalien]

Why split implicit messages from messages? IMO, we should just let you pass arbitrary messages from any sender (ideally by CID OR By explicitly specifying a message to execute).

TipSetMessageSelectionStrategy

We also need a way to say "and include all messages from this same sender in the message pool". Or even include all messages from these specific senders? All messages to the recipient?

It would be really nice to have a complex "message-inclusion-strategy" query here: {SameSender: bool, SameRecipient: bool, MessagePool: bool, TipSet: bool}.

(we can start with the simplest version, but I'd structure this as an object we can extend over time)

Also: does "all" mean "including implicit messages"? We probably need to distinguish between "all messages" and "all all".

debugging

I'd also add a "debug actors" option that enables FVM debugging and returns the logs (embedded in the trace).

[Kubuxu]

I don't think "from mpool" is a good idea, it is inherently racy and non-reproducible.

[masih]

Note to implementer: Also see #13046

[Stebalien]

I don't think "from mpool" is a good idea, it is inherently racy and non-reproducible.

I was thinking about things like gas estimation. But maybe that'll use a separate API? I'd prefer to have one API.