[NO NOT MERGE] Direct Nexus calls

[bergundy]

What changed?

Added two new RPCs for making direct Nexus requests using namespace credentials: StartNexusOperation and CancelNexusOperation.

There's an open PR (#685) that adds support for standalone Nexus operations (with APIs named StartNexusOperationExecution, etc...), which is different than this.
Direct calls do not instantiate a state machine on the caller namespace, they go directly to the handler that is registered for the given endpoint.

The API was modeled after https://github.com/nexus-rpc/api/blob/main/SPEC.md but uses Temporal concepts such a payloads instead Nexus contents. This API is intentionally not compatible with the HTTP spec.

Note that the HTTP routes explicitly qualify these APIs as "requests" to contrast standalone Nexus operation routes, which qualify as "operations". The semantics of the two are different enough that it was deemed the right approach to separate these APIs.

Why?

Allows backing UIs and CLIs with Nexus handlers running on Temporal workers or invoking synchronous system Nexus endpoint operations with minimal overhead.

Server PR

Not yet.

[cretz]

I was under the impression that operation token was only for handler-server and caller-server side, not caller client side which operates on operation IDs

[bergundy]

This is for exposing direct calls to a handler, bypassing the durable caller machinery. The semantics are different.

[cretz]

I don't understand. Where are the RPC calls for standalone Nexus operations from a client that are durable and take operation IDs and such? The name of these RPCs makes it seem like those. Or are standalone Nexus operations as being designed internally not occurring as designed?

[bergundy]

#685 see also PR description.

[cretz]

So both PRs have StartNexusOperation and mean different things, though both PR descriptions acknowledge the ambiguity.

I would recommend not using the term Execution to differentiate. I do think the outbound caller form should use Execution, but I think the "direct" form here should have a more qualifying name. I assume this is mostly for server-to-server because most people will want durable and therefore even if they are calling a handler on their own cluster, they'll use the durable/visibility-based form from the other PR, correct?

You're going to have to have an adjective to refer to this form of API regardless, so I recommend baking that adjective into the RPC names as you will with docs or anything else. Direct may be ok, but other options may make sense too, but we need some term to refer to these in both the RPC name and in conversation/docs IMO.

[bergundy]

The use cases are:

• Exposing system endpoint Nexus APIs to SDK clients (allows us to go all in on Nexus for public APIs, each API would automatically work within and outside of a workflow)
• Building interfaces with workers as a backend (e.g. for the UI and CLI)

I intentionally used the API names from the Nexus handler specification since there's a 1-1 mapping with those APIs.

[cretz]

I think we need a term for this approach and the RPCs should include that term to avoid ambiguity with the other ones. Execution isn't enough IMO to differentiate (and won't be in CLI, docs, or anywhere else).

[bergundy]

We are not planning on exposing these methods to the CLI or docs BTW. This is an internal detail that would be used inside the SDK. We have been using the terms Durable and Direct or Execution and Request.

I'm not sure what to call this. Maybe:

• StartDirectNexusOperation?
• CancelDirectNexusOperation?

Got better ideas?

[bergundy]

Alternatives:

• SendStartNexusOperationRequest
• SendCancelNexusOperationRequest