Standardize SDK configuration: introduce `workflow.config.ts` (RFC)

[pranaygp]

Summary

Workflow SDK configuration is currently spread across ~40 environment variables, typed plugin/createWorld options, and CLI flags — three uncoordinated surfaces with no single source of truth, inconsistent precedence rules, and no documentation page that lists them all. This issue inventories the current state and proposes a standardized, typed workflow.config.ts with a well-defined override hierarchy (defaults → workflow.config.ts → env var → CLI flag), plus a dedicated Configuration docs page.

This is a planning / RFC issue — feedback on the shape welcome before we commit to an API.

---

Problem: three config surfaces, no source of truth

Today, configuration is set in three different places depending on which knob you want to turn:

1. Environment variables — the bulk of configuration (WORKFLOW_*, plus VERCEL_*, DATABASE_URL, etc.). Untyped, undiscoverable, undocumented as a set.
2. Typed plugin / factory options — withWorkflow(nextConfig, { workflows: { lazyDiscovery, local: { port }, sourcemap } }), createVercelWorld(config), createPostgresWorld({ connectionString, pool, maxPoolSize, jobPrefix }), createLocalWorld({ dataDir, port, baseUrl }), and the builder BaseWorkflowConfig (dirs, projectRoot, externalPackages, distDir, sourcemap, …).
3. CLI flags — --backend, --project, --team, --env, --authToken, --json, --web, --webPort, --localUi, --port, …

Concrete pain points

• Precedence is inconsistent across the codebase. Two examples that contradict each other:

  • sourcemap: explicit config wins over env var (packages/builders/src/base-builder.ts — "config wins over env var, env var wins over the default").
  • lazyDiscovery: env var wins over the plugin option (packages/next/src/index.ts — "The WORKFLOW_NEXT_LAZY_DISCOVERY environment variable, if set, takes precedence over the option").
  • There is no documented, repo-wide rule for which source wins.

• Env vars are doing double duty as an internal transport. withWorkflow() translates its typed options into process.env (process.env.WORKFLOW_NEXT_LAZY_DISCOVERY = '1', process.env.WORKFLOW_TARGET_WORLD = 'local', process.env.WORKFLOW_LOCAL_DATA_DIR = '.next/workflow-data'). So env vars are simultaneously (a) a public user-facing config surface and (b) an internal IPC mechanism between the plugin and the runtime/builders. These two roles should be separated.

• Config options silently ignored. [nitro] workflow.dirs config option is ignored by VercelBuilder and LocalBuilder #1684 — workflow.dirs is ignored by the Vercel/Local builders. @workflow/nitro: steps bundle externalizes sibling workspace packages — no way to set projectRoot (breaks Nuxt/Nitro monorepo dev) #2074 — there is no way to set projectRoot, which breaks Nuxt/Nitro monorepo dev.

• No discoverability / no docs. There is no page that enumerates the configuration options. The only near-complete list lives implicitly in packages/web/app/server/workflow-server-actions.server.ts (a per-world env allowlist) and scattered package READMEs.

• Self-hosting friction & secret exposure. [web] Self-hosted deployment: UI doesn't reflect server-side config + auth token exposure risk #594 — self-hosted UI doesn't reflect server-side config, and auth tokens can be exposed.

• Feature requests blocked on config surface. feat - flow control & concurrency / parallelism limits #1206 (flow control / concurrency limits) effectively needs a real place to express per-workflow/per-deployment config.

---

Current state: full configuration inventory

Source-of-truth audit as of this issue. file:line references point at where each var is read.

Core runtime — @workflow/core

Env var	Controls	Default	Accepts
WORKFLOW_REPLAY_TIMEOUT_MS	Max wall-clock for the replay/VM portion of a single handler invocation. Invalid/out-of-range → default + warning.	240000 (clamped [30000, 780000])	int ms
WORKFLOW_V2_TIMEOUT_MS	Inline step-execution budget in the combined V2 handler before handoff to queue-based retry.	120000	int ms
WORKFLOW_TRACE_MODE	OTEL trace mode. Unrecognized values warn once and fall back.	linked	linked | continuous
DEBUG	Debug log filter (custom matcher, not the debug pkg). Supports wildcards + negation.	unset	e.g. workflow:*,-workflow:telemetry:*
WORKFLOW_TARGET_WORLD	Which world implementation to load.	auto: vercel if VERCEL_DEPLOYMENT_ID else local	local | vercel | @workflow/world-postgres | custom pkg/path

Platform-injected, read by core for detection: VERCEL, VERCEL_URL, VERCEL_DEPLOYMENT_ID, VERCEL_PROJECT_ID.

Queue (cross-cutting) — @workflow/world, @workflow/builders

Env var	Controls	Default
WORKFLOW_QUEUE_NAMESPACE	Queue topic namespace prefix (scopes topics to avoid cross-framework collisions). Must be ^[a-z][a-z0-9]*$.	none (__wkf_*)

World: local — @workflow/world-local

Env var	Controls	Default
WORKFLOW_LOCAL_DATA_DIR	Filesystem data dir for event log + run state.	.workflow-data (.next/workflow-data when set by @workflow/next)
WORKFLOW_LOCAL_BASE_URL	Base URL for the local queue + API endpoints (precedence over PORT).	http://localhost:3000 (CLI), else port autodetect
WORKFLOW_LOCAL_QUEUE_CONCURRENCY	Max concurrent local queue workers.	implementation default
WORKFLOW_LOCAL_QUEUE_MAX_VISIBILITY	Max visibility timeout for local queue messages.	implementation default
PORT	Fallback port for base-URL resolution.	—

World: Vercel — @workflow/world-vercel

Env var	Controls	Notes
WORKFLOW_VERCEL_ENV	Target Vercel environment.	tooling-side (not read at runtime inside the function)
WORKFLOW_VERCEL_TEAM	Team ID.	tooling-side
WORKFLOW_VERCEL_PROJECT	Project ID.	tooling-side
WORKFLOW_VERCEL_PROJECT_NAME	Project slug/name.	tooling-side
WORKFLOW_VERCEL_AUTH_TOKEN	Vercel auth token (secret).	tooling-side
WORKFLOW_VERCEL_BACKEND_URL	Custom backend proxy URL.
VERCEL_WORKFLOW_SERVER_URL	Workflow server URL (deployment-time).
VERCEL_QUEUE_MAX_DELAY_SECONDS	Max queue delay.	82800 (23h)
VERCEL_PROJECT_ID	projectId fallback when not in config.	platform
VERCEL_TOKEN	Auth fallback for external tooling/o11y (secret).
VERCEL_DEPLOYMENT_KEY	Deployment encryption key (secret).	platform

World: Postgres — @workflow/world-postgres

Env var	Controls	Default
WORKFLOW_POSTGRES_URL	Connection string.	postgres://world:world@localhost:5432/world
DATABASE_URL	Connection string fallback (used after WORKFLOW_POSTGRES_URL).	—
WORKFLOW_POSTGRES_JOB_PREFIX	Queue job-name prefix.	—
WORKFLOW_POSTGRES_WORKER_CONCURRENCY	Concurrent workers.	50
WORKFLOW_POSTGRES_MAX_POOL_SIZE	Internal pg.Pool max size (when pool omitted).	10

Community worlds (referenced via worlds-manifest.json)

WORKFLOW_TURSO_DATABASE_URL; WORKFLOW_MONGODB_URI, WORKFLOW_MONGODB_DATABASE_NAME; WORKFLOW_REDIS_URI; JAZZ_API_KEY, JAZZ_WORKER_ACCOUNT, JAZZ_WORKER_SECRET.

Build / compiler (cross-framework) — @workflow/builders + adapters

Env var	Controls	Default / precedence
WORKFLOW_SOURCEMAP	Sourcemap mode for generated bundles (true/inline/linked/external/both/false).	inline; explicit config > env > default
WORKFLOW_EMIT_SOURCEMAPS_FOR_DEBUGGING	Legacy back-compat sourcemap toggle.	=1
WORKFLOW_PUBLIC_MANIFEST	Expose the workflow manifest as a public HTTP route / static file. Required for e2e.	=1
WORKFLOW_QUEUE_NAMESPACE	(resolved at build time into generated routes)	see above

Next.js integration — @workflow/next

Env var	Controls	Default
WORKFLOW_NEXT_LAZY_DISCOVERY	Defer workflow discovery until first access. env > plugin option > default.	true
WORKFLOW_NEXT_PRIVATE_BUILT	Internal double-build guard.	—
WORKFLOW_NEXT_DIST_DIR	Override Next dist dir.	.next
WORKFLOW_PROJECT_ROOT	Project root for module resolution.	—
WORKFLOW_SOCKET_INFO_PATH / WORKFLOW_SOCKET_PORT / WORKFLOW_SOCKET_AUTH	loader↔builder IPC (auto-managed).	auto
WATCHPACK_WATCHER_LIMIT	macOS watcher cap (auto-set to 20).	—
TURBOPACK, NODE_ENV, NEXT_RUNTIME, PORT	platform/runtime detection.	—

CLI — @workflow/cli

Env: WORKFLOW_NO_UPDATE_CHECK, WORKFLOW_JSON_MODE (set by --json), WORKFLOW_OBSERVABILITY_CWD, WORKFLOW_LOCAL_UI (set by --localUi), WORKFLOW_MANIFEST_PATH.

Flags: --verbose/-v, --json/-j, --cursor, --backend/-b, --authToken/-a, --project, --team, --env/-e, --web/-w, --webPort, --noBrowser, --localUi, --sort, --port.

Web UI — @workflow/web

WORKFLOW_WEB_VERCEL_BUILD (toggles Vercel preset in the web build), WORKFLOW_EMBEDDED_DATA_DIR (legacy embedded data dir), plus shared WORKFLOW_MANIFEST_PATH, WORKFLOW_OBSERVABILITY_CWD.

CI detection (generic, terminal formatting only)

CI, CONTINUOUS_INTEGRATION, BUILD_NUMBER, GITHUB_ACTIONS, GITLAB_CI, CIRCLECI, TRAVIS, JENKINS_URL, BUILDKITE, TF_BUILD.

---

Existing precedence models worth standardizing on

The repo already has good local precedence chains we can generalize:

• Local base URL (world-local/src/config.ts): config.baseUrl → WORKFLOW_LOCAL_BASE_URL → config.port → PORT → autodetect.
• Postgres pool (world-postgres): createWorld({ maxPoolSize }) → WORKFLOW_POSTGRES_MAX_POOL_SIZE → pg default.
• Sourcemap (builders): explicit config → WORKFLOW_SOURCEMAP → default.

The proposal picks one rule and applies it everywhere.

---

Proposal: workflow.config.ts

A single, typed, framework-agnostic config file at the project root, loaded once and consumed by core, builders, the active world, and the CLI.

Shape

// workflow.config.ts
import { defineConfig } from 'workflow/config';

export default defineConfig({
  // ── Which backend ("world") runs your workflows ──────────────────
  world: {
    // 'local' | 'vercel' | '@workflow/world-postgres' | custom package/path
    target: 'local',

    // Per-world options are typed by the chosen target (discriminated union).
    local: {
      dataDir: '.workflow-data',
      baseUrl: 'http://localhost:3000',
      queue: { concurrency: 10, maxVisibilityMs: 30_000 },
    },
    // vercel: { env, team, project, projectName, backendUrl, serverUrl, queue: { maxDelaySeconds } }
    // postgres: { connectionString, jobPrefix, workerConcurrency, maxPoolSize }
    //
    // Secrets (auth tokens, connection strings) are NOT inlined here — they
    // stay in env vars and are referenced, never committed. See "Secrets".
  },

  // ── Runtime behavior (@workflow/core) ────────────────────────────
  runtime: {
    replayTimeoutMs: 240_000,   // WORKFLOW_REPLAY_TIMEOUT_MS
    inlineStepTimeoutMs: 120_000, // WORKFLOW_V2_TIMEOUT_MS
    traceMode: 'linked',        // 'linked' | 'continuous'
    debug: 'workflow:*',        // DEBUG filter
  },

  // ── Queue ─────────────────────────────────────────────────────────
  queue: {
    namespace: 'myapp',         // WORKFLOW_QUEUE_NAMESPACE
  },

  // ── Build / compiler (@workflow/builders + framework adapters) ────
  build: {
    dirs: ['workflows'],        // where to discover workflow files
    projectRoot: '.',           // resolves #2074
    externalPackages: [],
    distDir: '.next',
    sourcemap: 'inline',        // WORKFLOW_SOURCEMAP
    publicManifest: false,      // WORKFLOW_PUBLIC_MANIFEST
    next: { lazyDiscovery: true },
  },

  // ── Observability / web UI ────────────────────────────────────────
  observability: {
    localUi: false,
    webPort: 3456,
    manifestPath: undefined,
  },
});

defineConfig is a pure identity helper that gives full type inference + autocomplete and lets us evolve the schema with deprecation warnings. The file is TS-first (matching the platform's move to vercel.ts), but a workflow.config.js/.mjs and a "workflow" key in package.json should also be accepted.

Precedence (single repo-wide rule)

From lowest to highest priority:

built-in defaults
  → workflow.config.ts
    → environment variables          (WORKFLOW_*, etc.)
      → explicit CLI flags / plugin args

Rationale: the file is the durable, committed source of truth; env vars override it for per-environment differences (CI, preview vs prod, secrets) without editing code; explicit CLI flags / call-site plugin args win because they are the most local and intentional. This is a deliberate, single decision that replaces today's contradictory sourcemap (config>env) vs lazyDiscovery (env>config) behavior.

Note this flips the current sourcemap behavior (today config beats env). We'd document this as a breaking change for that one knob and keep a one-release shim.

Every option keeps an env override (documented mapping)

The config file does not remove env vars — it gives them a typed home and a documented 1:1 mapping. Every field has a canonical env var:

Config path	Env override
runtime.replayTimeoutMs	WORKFLOW_REPLAY_TIMEOUT_MS
runtime.inlineStepTimeoutMs	WORKFLOW_V2_TIMEOUT_MS
runtime.traceMode	WORKFLOW_TRACE_MODE
world.target	WORKFLOW_TARGET_WORLD
world.local.dataDir	WORKFLOW_LOCAL_DATA_DIR
build.sourcemap	WORKFLOW_SOURCEMAP
build.publicManifest	WORKFLOW_PUBLIC_MANIFEST
queue.namespace	WORKFLOW_QUEUE_NAMESPACE
…	… (full table generated from the schema)

A code-generation step can emit this table into the docs from the schema so it never drifts.

CLI arg overrides

CLI flags map onto the same schema and sit at the top of the precedence chain, e.g. workflow inspect --backend vercel --project … --env preview overrides world.target/world.vercel.* for that invocation only. Flags that are purely presentational (--json, --verbose, --web) stay flags-only.

Secrets

Connection strings and auth tokens (WORKFLOW_VERCEL_AUTH_TOKEN, WORKFLOW_POSTGRES_URL, VERCEL_TOKEN, WORKFLOW_REDIS_URI, …) are never inlined in workflow.config.ts. They remain env-only. The config can reference them, and the schema marks fields as secret so the web UI / inspect know to redact them (ties into #594). The existing WORLD_SENSITIVE_ENV_KEYS set in workflow-server-actions.server.ts becomes derived from the schema.

Loading mechanism

• A small workflow/config loader (in @workflow/core or a new @workflow/config) resolves + validates the file once, merges defaults/env/flags per the precedence rule, and exposes a typed resolveConfig().
• Replaces the withWorkflow() → process.env bridge. Today the Next plugin stuffs typed options into process.env so downstream builders/runtime can read them. With a real config object, that internal transport goes away — frameworks read the resolved config directly. Env vars go back to being only a user-facing override surface, not an IPC channel.
• Framework adapters (next, nitro, sveltekit, astro, nuxt, vite, nest, hono, express, fastify) read the resolved config; their existing plugin options remain supported and map to the same fields (call-site args win, per precedence).

---

Docs: a dedicated "Configuration" page

New page under docs/content/docs/<version>/ (e.g. foundations/configuration or a top-level configuration):

• The full workflow.config.ts reference (every field, type, default).
• The generated config-path → env-var mapping table.
• The precedence rule, stated once, with examples.
• Per-world configuration sections (local / vercel / postgres / community).
• Secrets guidance.
• Migration guide from "env vars only" → workflow.config.ts.

Keep the existing agent-discoverable docs sitemap behavior intact (per repo CLAUDE.md).

---

Rollout plan (non-breaking first)

1. Schema + loader + defineConfig in a new @workflow/config (or workflow/config subpath). No behavior change yet — resolveConfig() reads defaults + existing env vars so current setups keep working.
2. Adopt internally: route core/builders/worlds reads through resolveConfig() instead of direct process.env access. Standardize the precedence rule here. Keep all env vars working.
3. Wire frameworks: have adapters consume the resolved config; remove the withWorkflow → process.env internal bridge (options still accepted).
4. CLI: flags feed the same resolver.
5. Docs: ship the Configuration page + generated mapping table.
6. Deprecations: warn on the few behavior changes (e.g. sourcemap precedence flip, WORKFLOW_EMIT_SOURCEMAPS_FOR_DEBUGGING → build.sourcemap, WORKFLOW_EMBEDDED_DATA_DIR → world.local.dataDir). One-release shims.

Backwards compatibility is the priority: workflow.config.ts is additive. Projects with zero config and only env vars must continue to work unchanged.

---

Open questions

• Package home: new @workflow/config vs a workflow/config subpath export? (Leaning subpath to avoid an extra install.)
• File formats: TS-only, or also .js/.mjs/package.json#workflow? TS needs a loader (jiti/bundle) at build & CLI time.
• Precedence for lazyDiscovery/sourcemap: confirm we're OK flipping sourcemap to env-over-config for consistency, or keep per-field exceptions documented.
• Per-workflow config: does this file also become the home for feat - flow control & concurrency / parallelism limits #1206 (concurrency / flow-control limits), or is that a separate per-workflow construct?
• Validation: zod/valibot vs hand-rolled — affects bundle size for the runtime path.
• Monorepo resolution: how the loader finds the config from nested apps (relates to @workflow/nitro: steps bundle externalizes sibling workspace packages — no way to set projectRoot (breaks Nuxt/Nitro monorepo dev) #2074).

---

Related issues

• [nitro] workflow.dirs config option is ignored by VercelBuilder and LocalBuilder #1684 — workflow.dirs config ignored by Vercel/Local builders (config option not honored).
• @workflow/nitro: steps bundle externalizes sibling workspace packages — no way to set projectRoot (breaks Nuxt/Nitro monorepo dev) #2074 — no way to set projectRoot; breaks Nuxt/Nitro monorepo dev (missing config surface).
• [web] Self-hosted deployment: UI doesn't reflect server-side config + auth token exposure risk #594 — self-hosted UI doesn't reflect server-side config + token exposure (config visibility/secrets).
• feat - flow control & concurrency / parallelism limits #1206 — flow control & concurrency limits (feature blocked on a real config home).

Inventory compiled from a full process.env audit of packages/* (excluding tests/e2e/CHANGELOGs). file:line references available on request.