Is your feature request related to a problem? Please describe.
Temporal Schedules provide a durable model for when to start a Workflow, and
Workflow Executions provide a durable model for how to run it. A recurring
application pattern is still left to every user to build: running the same
scheduled Workflow over a large, mutable set of subscribed entities while
freezing the exact membership used by each occurrence.
Examples include polling devices, refreshing tenant data, recurring billing,
inventory synchronization, monitoring targets, campaign audiences, and data
collection. Items are added, paused, resumed, or removed while the Schedule is
active.
Applications generally choose one of these approaches:
- Create one Schedule or long-lived Workflow per entity.
- Maintain a long-lived coordinator Workflow through Signals or Updates.
- Keep membership in an external database and have a scheduled dispatcher
Workflow query, snapshot, paginate, fan out, deduplicate, and reconcile it.
The third approach is often the most practical, but it creates a dual
consistency boundary between Temporal and the external subscription store.
Every application must independently implement membership revisions, snapshot
isolation, occurrence identity, duplicate suppression, pagination, crash
recovery, reconciliation, and visibility.
A concrete example is a daily Workflow over 10,000 collection targets. The set
may change while the Schedule fires, but every occurrence must use one exact,
explainable snapshot. Pagination must not skip or duplicate items because the
external set changed during planning, and a process crash must not create a
second logical execution of the same item for that occurrence.
Describe the solution you'd like
Please consider a first-class durable subscription set (or versioned target
set) that can be bound to a Schedule action. The important part is the
semantics rather than a specific API:
- A named set has a monotonic membership revision and supports idempotent item
add, update, pause, resume, and remove operations.
- A Schedule occurrence freezes one exact membership revision. Concurrent
mutations affect either that occurrence or a future one, never an ambiguous
partial set.
- The occurrence exposes a stable identity and frozen set revision so item or
child Workflow IDs can be derived deterministically.
- Large sets are consumed through bounded pages or manifests instead of being
copied into one Workflow input or History event.
- Recovery of the same occurrence converges on the same logical item set and
does not silently omit or duplicate members.
- Set state, frozen revision, occurrence progress, and failures are visible
through Temporal APIs and, eventually, the UI.
- Items only require a stable key and an opaque bounded payload or external
data reference; Temporal does not need to understand business data.
One possible execution model is:
Schedule
-> freeze SubscriptionSet revision R for occurrence O
-> start coordinator Workflow with (set ID, R, O)
-> read bounded membership pages
-> start or signal deterministic item/child Workflows
This is not a request for exactly-once effects in external systems. It is a
request for one durable consistency model covering Schedule occurrence creation
and the target-set revision selected by that occurrence.
If a server primitive is considered too broad, an official cross-SDK
abstraction or reference pattern with explicit snapshot, pagination,
Continue-As-New, deterministic identity, and recovery semantics would still be
valuable.
Describe alternatives you've considered
- One Schedule per entity: simple semantics, but high operational
cardinality and poor support for bulk lifecycle changes and batch execution.
- One long-lived entity Workflow per item: useful for entity lifecycles, but
not always appropriate for a periodically changing batch population.
- A coordinator Workflow with Signals/Updates: possible, but every user must
design History rollover, membership revisioning, paging, and visibility.
- External database plus dispatcher Workflow: scalable, but requires a
custom dual-write/idempotency protocol and reconciliation between the
external store and Temporal.
Additional context
This pattern applies to billing populations, IoT fleets, monitoring targets,
tenant synchronization, inventory refresh, campaigns, and data pipelines. A
Temporal-owned primitive or canonical abstraction could prevent these systems
from repeatedly building incompatible subscription control planes around the
service.
Related proposals with different scopes:
Neither appears to cover versioned target-set membership frozen per Schedule
occurrence.
I can provide a concrete application use case and help clarify the expected
semantics and failure scenarios, but I am not currently able to implement this
server-level feature.
Is your feature request related to a problem? Please describe.
Temporal Schedules provide a durable model for when to start a Workflow, and
Workflow Executions provide a durable model for how to run it. A recurring
application pattern is still left to every user to build: running the same
scheduled Workflow over a large, mutable set of subscribed entities while
freezing the exact membership used by each occurrence.
Examples include polling devices, refreshing tenant data, recurring billing,
inventory synchronization, monitoring targets, campaign audiences, and data
collection. Items are added, paused, resumed, or removed while the Schedule is
active.
Applications generally choose one of these approaches:
Workflow query, snapshot, paginate, fan out, deduplicate, and reconcile it.
The third approach is often the most practical, but it creates a dual
consistency boundary between Temporal and the external subscription store.
Every application must independently implement membership revisions, snapshot
isolation, occurrence identity, duplicate suppression, pagination, crash
recovery, reconciliation, and visibility.
A concrete example is a daily Workflow over 10,000 collection targets. The set
may change while the Schedule fires, but every occurrence must use one exact,
explainable snapshot. Pagination must not skip or duplicate items because the
external set changed during planning, and a process crash must not create a
second logical execution of the same item for that occurrence.
Describe the solution you'd like
Please consider a first-class durable subscription set (or versioned target
set) that can be bound to a Schedule action. The important part is the
semantics rather than a specific API:
add, update, pause, resume, and remove operations.
mutations affect either that occurrence or a future one, never an ambiguous
partial set.
child Workflow IDs can be derived deterministically.
copied into one Workflow input or History event.
does not silently omit or duplicate members.
through Temporal APIs and, eventually, the UI.
data reference; Temporal does not need to understand business data.
One possible execution model is:
This is not a request for exactly-once effects in external systems. It is a
request for one durable consistency model covering Schedule occurrence creation
and the target-set revision selected by that occurrence.
If a server primitive is considered too broad, an official cross-SDK
abstraction or reference pattern with explicit snapshot, pagination,
Continue-As-New, deterministic identity, and recovery semantics would still be
valuable.
Describe alternatives you've considered
cardinality and poor support for bulk lifecycle changes and batch execution.
not always appropriate for a periodically changing batch population.
design History rollover, membership revisioning, paging, and visibility.
custom dual-write/idempotency protocol and reconciliation between the
external store and Temporal.
Additional context
This pattern applies to billing populations, IoT fleets, monitoring targets,
tenant synchronization, inventory refresh, campaigns, and data pipelines. A
Temporal-owned primitive or canonical abstraction could prevent these systems
from repeatedly building incompatible subscription control planes around the
service.
Related proposals with different scopes:
Event History events and notification triggers.
registration and synchronization of Schedules.
Neither appears to cover versioned target-set membership frozen per Schedule
occurrence.
I can provide a concrete application use case and help clarify the expected
semantics and failure scenarios, but I am not currently able to implement this
server-level feature.