fix(workers): resolve worker deployment from task queue - #3883
rossnelson wants to merge 2 commits into
Conversation
The workflow page reads the worker deployment from the TemporalWorkerDeployment search attribute. The server only writes that attribute once a versioned worker completes the first workflow task, so during a compute-provider cold start it is empty. Two things follow: the no-workers alert falls through to the self-managed branch and tells the user to check Kubernetes or ECS, and the Deployment column in the detail row renders blank. Resolve the deployment from the task queue instead. The task queue response the page already fetches carries versioningInfo.currentDeploymentVersion.deploymentName, which is a property of the queue rather than the workflow, so it is present with zero pollers and needs no extra request. The search attribute stays as a fallback for workflows pinned to a version that is no longer current. Guard on currentDeploymentVersion, not on versioningInfo. A task queue with no registered version still returns a versioningInfo object containing currentVersion "__unversioned__", so a truthiness check on versioningInfo matches every unversioned queue. Also fixes the alert on the standalone activity page, which never rendered at all: reducePollerTypes always returns an array, so `!response.pollers` was always false. It now checks length, matching isRunningWithNoWorkers. That layout also passed no deployment, so it could only ever show the self-managed message. Removes workflow-error-no-compatible-workers-title/-description, which no component references. Verified against a local server with workercontroller.enabled and a real AWS Lambda serverless worker: one DescribeTaskQueue response returned `pollers: []` alongside currentDeploymentVersion while the search attribute was still empty. Refs FE-489
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
| workflow: WorkflowExecution | null | undefined, | ||
| ): string | undefined => | ||
| workers?.versioningInfo?.currentDeploymentVersion?.deploymentName || | ||
| workflow?.searchAttributes?.indexedFields?.['TemporalWorkerDeployment'] || |
There was a problem hiding this comment.
The search attribute stays as a fallback for workflows pinned to a version that is no longer current.
Just want to verify that the task queue's current deployment version should always take precedence over the workflow's deployment search attribute.
There was a problem hiding this comment.
Good question, and no, it should not. Current Version routes new executions and existing unversioned or AutoUpgrade workflows only (see the doc on TaskQueueVersioningInfo.current_deployment_version). A pinned workflow keeps its deployment after the queue moves on, and the server records that deployment in TemporalWorkerDeployment. Reading the queue first would have shown the wrong deployment for a pinned workflow, beside Version and Behavior columns that still read the attribute.
Swapped the order in 5ea898b: the attribute comes first, and the task queue fills the gap before the first workflow task completes, which is the cold-start window this PR exists for. Added a test for a workflow pinned to a deployment that is no longer current, and fixed the sentence in the description.
The task queue's Current Version routes new executions and unversioned or AutoUpgrade workflows. A pinned workflow keeps its own deployment after the queue moves on, and the server records that deployment in the TemporalWorkerDeployment search attribute. Reading the queue first would show the wrong deployment for such a workflow, beside Version and Behavior columns that still read the attribute. The attribute now comes first. The task queue still fills the gap before the first workflow task completes, which is the cold-start window this fix exists for.
2 participants
Problem
When a workflow runs on a task queue served by a Worker Deployment with a compute provider, the worker has not cold-started yet. A user opening the workflow page in that window sees:
Every actionable sentence is wrong. There is no Kubernetes or ECS, no worker logs to read, and nothing to restart — Temporal is invoking the customer's Lambda on their behalf.
Cause
workflow-header.svelteread the deployment from theTemporalWorkerDeploymentsearch attribute. The server only writes that attribute once a versioned worker completes the first workflow task, so during a cold start it is empty andno-workers-polling-alert.sveltefalls through to the self-managed branch. The calm serverless message on the other branch was only reachable on a second scale-down — never on a first run.workflow-details.svelteread the same attribute, so the Deployment / Build ID / Versioning Behavior column was blank at the same moment.Fix
Fall back to the task queue when the attribute is empty. The response the page already fetches carries
versioningInfo.currentDeploymentVersion.deploymentName— a property of the queue, not the workflow, so it is present with zero pollers and needs no extra request.The search attribute still comes first when present. It records the deployment the workflow actually runs on, and a pinned workflow keeps it after the queue's Current Version moves to another deployment. Current Version only routes new executions and unversioned or AutoUpgrade workflows, so reading the queue first would show the wrong deployment for a pinned workflow.
Extracted to
getWorkerDeploymentName()rather than repeating the optional-chain across three call sites.Guard on
currentDeploymentVersion, notversioningInfoA task queue with no registered version still returns a
versioningInfoobject:{ "currentVersion": "__unversioned__", "updateTime": "0001-01-01T00:00:00Z" }A truthiness check on
versioningInfotherefore matches every unversioned queue and would pick the wrong branch for all of them. There is a test covering this.Also fixed: an alert that never rendered
standalone-activity-layout.sveltehad:reducePollerTypesalways returns an array from.map(), so!response.pollersis![]→ always false. That alert has never rendered on the standalone activity page under any conditions. Now checks.length, matching whatisRunningWithNoWorkersalready does.That layout also passed no
deploymentprop at all, so even once visible it could only ever show the self-managed message.Also removed
workflow-error-no-compatible-workers-title/-description— no component references either.Verification
Against a local server (1.31.2) with
workercontroller.enabledand a real AWS Lambda serverless worker invoked through the worker controller:DescribeTaskQueueresponse returnedpollers: []alongsidecurrentDeploymentVersion: {ross-test, v1}, via the same HTTP path the UI callsTemporalWorkerDeploymentwas confirmed empty on the running workflow at that momentChecks: ESLint clean,
svelte-checkclean, 5 new unit tests pass. Full suite is 2767 passing — the 26 failing test files are pre-existing@buf/@bufbuildmodule-resolution failures present onmain(verified by stashing).Deliberately not in this PR
The messaging rework is blocked on a product decision and would be rebuilt if it landed early. A correct cold start resolves in ~2.4s, so a "Starting a worker" message would appear and vanish faster than anyone can read — arguably worse than showing nothing. That decision changes the shape of the state model, so it is tracked separately on FE-489.
Also excluded: a
provisioning-failedstate.providerValidationis absent from the API response even for a deployment that passed real AWS validation, soconnection-status.tsderivespendingfor a healthy deployment. There is no data source to build it on yet.This PR is a strict improvement regardless of how those land: the right message fires, and the Deployment column populates.