Fire TableTriggers when an upstream View is updated

[ryannedolan]

Summary

• Add .status.watermark to View objects.
• Add ViewReconciler to detect changes to view watermarks.
• Automatically fire TableTriggers when an upstream View is updated.

Details

In order to support jobs that listen to multiple tables, we have a few options:

1. Make breaking changes to TableTrigger. Instead of one table, listen to an array of tables.
2. Add a second Trigger type that supports multiple tables. Maybe call it MultiTableTrigger.
3. Leverage View, which supports an arbitrary number of source tables.

This PR implements option 3, which is probably the least obvious. The motivation here is to mimic traditional CREATE TRIGGER DDL, e.g. as supported by Oracle, which works with either tables or views but not lists of tables. In typical DDL, you must first CREATE VIEW FOO and then CREATE TRIGGER BAR ON FOO. This means the trigger has a single object to listen to (the view object) rather than a list of objects.

In order to support this pattern, I've added a new ViewRconciler which monitors .status.watermark and fires any listening TableTriggers. Batch-oriented jobs that materialize a view can update the view's watermark to trigger downstream jobs. Stream-oriented jobs can do the same whenever they checkpoint.

Testing Done

New unit test passes:

Reconciling request Request{name='view-1', namespace='namespace'}
View view-1 was updated at 2025-11-06T14:10:11.309811-06:00.
Firing downstream trigger table-trigger.

[jogrogan]

Is the correlation between objects, in terms of ownership still
View -> Pipeline -> Trigger?

It seems odd to now have a link from the Trigger back to the View

[ryannedolan]

The View here isn't necessarily the same View as in your example. In the typical View->Pipeline->Trigger situation, the Trigger likely listens to one of the source tables in the View, not the View itself. Or, the Trigger may listen to an upstream View, if you have a views-of-views situation.

I'm planning to add a CREATE TRIGGER DDL along these lines:

CREATE MATERIALIZED VIEW FOO AS SELECT a, b FROM T;
CREATE TRIGGER BAR ON FOO AS 'my-job-template';

In this case, the CREATE...VIEW may result in Triggers, but they would listen to specific tables, not views. The CREATE TRIGGER would create a new Trigger that is wired up to listen to the View object, rather than a set of physical tables. Both leverage JobTemplates and TableTriggers under the hood, but in slightly different ways.

[jogrogan]

Great, this makes sense. Seems pretty powerful too, seems like we can define multiple triggers on the same view that may do different things at different cadences or whatnot as we expand the trigger feature set.

[srnand]

The table field is currently a required property in this CRD. Now that the view will have its own reconciler, which will handle updates and subsequently update the trigger timestamp in the table trigger object, do we still need the viewRef?

Instead, we could introduce a breaking change by renaming the table property to something more generic, such as source, which could represent either a single table or a view.

[ryannedolan]

I thought of that, but a view is technically a kind of table (especially if it's materialized), so I think it's reasonable to reuse table here.

[srnand]

yep, that works! I was mainly thinking that we don’t need a separate view ref.

[ryannedolan]

Yeah I'm going to drop the viewRef and use a label instead. That way the new ViewReconciler can fire any TableTriggers that have a matching view label.

[srnand]

looks like one of the integration tests failed ? otherwise lgtm

[jogrogan]

I imagine this view label will be added by the CREATE TRIGGER statement, right?

[ryannedolan]

yes that's the plan

[jogrogan]

LGTM