What happens if multiple workflow instances share the same hook token?

[sashkode]

• Can I use the same token when creating hooks in multiple workflow instances?
• What happens if I resume a hook with a token that is shared by multiple workflows?

Docs say:

Include routing information: Ensure the token contains enough information to identify the correct workflow instance

..but I'm curious if they can purposefully be the same.

This would open the door to notifying multiple workflow instances at once that they should resume. I'm specifically looking into building some sort of real-time/live query infra on top of workflows, and the ability to invalidate queries through hooks causing the query to rerun and stream the next value would be 🔥. Alternative, would seem to be to store instances (and their respective tokens) in some sort of KV, and loop through them to resume. Doesn't scale well, so curious what you guys think.

PS: As a product guy, I'm super excited about using workflows as a new primitive for building cool software, amazing tool.

[pranaygp]

Yeah we made an intentional choice to have a 1:1 mapping between a token and a workflow run. It enables things like respondWith in the webhook abstraction. We went through multiple iterations on the design before arriving at this constraint.

If you try to register the same token twice, it should throw a conflict error in your workflow (if it's not failing, that's a bug 😬)

To resume multiple runs from one event, we kinda leave it upto you rn to either store or calculate all the possible tokens you want to resume and then resume them all at once using Promise.all (btw you can also just attempt to resumeHook for tokens that don't exist, and then eat up and silently ignore the error. So if the cardinality on your token space is small, I recommend that approach)

The Tags RFC #132 would be a way where you can set a "tag" when creating the hook so that it's easier to just query all the hooks that have the same tags, and resume them all more easily

[pranaygp]

Also I would LOVE to have an example in our docs for this pattern if you end up implementing it and I would really appreciate your help 🙏

[sashkode]

Totally makes sense for the respondWith, I had not considered. Currently (at least in the local world), it's letting me create the hooks no problem, but the resume always targets the first workflow that created it. I added a super minimal repro below.

Tags seem like a powerful primitive, I love the potential for 'rolling your own idempotency'. I'll likely comment more on the RFC once I think a bit about it.

As for the live/real-time queries POC, I've got a version going, but it uses a workflow as a workflow instance registry. It's a bit messy, and kinda just handles subscribers and calling resume like a pub/sub. Would probably be better to just use the new Vercel Queues, any insight on the readiness of the package?

I'll follow up, once I have the use case nailed down. FYI, I'm also looking into using workflows for Stripe and maybe some hand rolled subscription billing.

Anyway, here's the minimal repro:

Workflow:

import { createHook, getWorkflowMetadata } from "workflow";

const logStart = async (runId: string) => {
  "use step";
  console.log(`Workflow started: ${runId}`);
};

const logResume = async (runId: string) => {
  "use step";
  console.log(`Workflow resumed: ${runId}`);
};

export const wait = async () => {
  "use workflow";

  const runId = getWorkflowMetadata().workflowRunId;

  await logStart(runId);

  const hook = createHook({
    token: "resume",
  });

  await hook;
  await logResume(runId);
};

Actions (Triggers):

"use server";

import { resumeHook, start } from "workflow/api";
import { wait } from "./(workflows)/wait";

export const startWorkflow = async function () {
  const run = await start(wait);
  return run.runId;
};

export const resumeAll = async function () {
  await resumeHook("resume", []);
};

And finally the logs I get when starting 3 instances, then calling resume twice:

 POST / 200 in 74ms (compile: 7ms, render: 67ms)
 POST /.well-known/workflow/v1/flow 200 in 533ms (compile: 385ms, render: 148ms)
Workflow started: wrun_01K9NQ5W937CBFGQCRZRR2NQFN
 POST /.well-known/workflow/v1/step 200 in 297ms (compile: 163ms, render: 134ms)
 POST /.well-known/workflow/v1/flow 200 in 99ms (compile: 1245µs, render: 98ms)
 POST / 200 in 18ms (compile: 4ms, render: 14ms)
 POST /.well-known/workflow/v1/flow 200 in 155ms (compile: 1766µs, render: 153ms)
Workflow started: wrun_01K9NQ5Z354PN2MVHFQ3ENB74Y
 POST /.well-known/workflow/v1/step 200 in 103ms (compile: 3ms, render: 100ms)
 POST /.well-known/workflow/v1/flow 200 in 103ms (compile: 1538µs, render: 102ms)
 POST / 200 in 19ms (compile: 5ms, render: 14ms)
 POST /.well-known/workflow/v1/flow 200 in 149ms (compile: 1446µs, render: 148ms)
Workflow started: wrun_01K9NQ60MTWR6TG2P5HE6GTA3M
 POST /.well-known/workflow/v1/step 200 in 102ms (compile: 1242µs, render: 101ms)
 POST /.well-known/workflow/v1/flow 200 in 102ms (compile: 1366µs, render: 101ms)
 POST / 200 in 21ms (compile: 4ms, render: 18ms)
 POST /.well-known/workflow/v1/flow 200 in 155ms (compile: 1201µs, render: 154ms)
Workflow resumed: wrun_01K9NQ5W937CBFGQCRZRR2NQFN
 POST /.well-known/workflow/v1/step 200 in 100ms (compile: 1082µs, render: 99ms)
 POST /.well-known/workflow/v1/flow 200 in 101ms (compile: 1226µs, render: 100ms)
 POST / 200 in 12ms (compile: 2ms, render: 10ms)
Workflow "wrun_01K9NQ5W937CBFGQCRZRR2NQFN" has status "completed", skipping
 POST /.well-known/workflow/v1/flow 200 in 3ms (compile: 863µs, render: 2ms)