feat(workflows): dispatch eels runs when client images update

[spencer-tb]

The eels simulators now run on client image updates, instead of on daily crons — the eels crons are removed, and git-source builds become a manual-dispatch-only option.

Supersedes #82: with the crons gone there is no schedule path to flip to docker — every dispatch (watcher or manual) already defaults to the prebuilt pandaops images.

When does a run happen?

• A client team pushes to their devnet branch → when pandaops rebuilds their image (hourly), that devnet's tests run for that client (full and quick workflows).
• A client team pushes to master/main → the eels tests run for that client on generic (consume-engine, consume-rlp, consume-enginex).
• We change what's tested — a new fixtures release or a different eels branch in a workflow file → that workflow re-runs for all clients.
• Manual dispatch works as before, including client_source=git for tip-of-branch builds — that's now the only way git builds run.
• Nothing else starts runs: eels or hive commits alone don't, the non-eels sim wrappers (devp2p, consensus, sync, …) keep their daily crons, and clients that haven't pushed a devnet branch yet are simply skipped.

Questions you'll have

• What if the client is already mid-run when its image updates? The new run queues behind it (per-client+simulator concurrency group). More updates while queued just replace the one pending run — a client is never tested twice concurrently, and whichever run starts pulls the newest image, since the tag resolves at job start.
• Won't busy clients spam runs? Dispatch rate is bounded by their commit rate (images rebuild at most hourly). Superseded pending runs show up as cancelled entries in the Actions tab but never consume a test runner.
• What happens right after merge? The state file starts empty, so the first tick dispatches every workflow once for every client with an existing image. It converges immediately after.
• What if Docker Hub/ghcr or a dispatch fails? A failed digest lookup skips that client for one tick. State is persisted only after all dispatches succeed, so a failed tick re-dispatches next tick — duplicates collapse in the concurrency queues. Failure mode is an extra run, never a missed one.
• Why not also trigger on eels/hive pushes? Deliberate: branch tips move constantly and each re-dispatch is all-clients. Only file edits (fixtures/branch/hive pins) re-dispatch.
• What do we do at the next devnet bump? Nothing: tags, clients, and images are derived from the workflow files, so the usual bump PR retargets the watcher automatically.
• What replaces the crons' retry role? If a run is lost to an infra outage, re-dispatch it manually (or wait for the next change). If that proves annoying in practice, a "re-dispatch when a client has no recent results" rule can be added to the watcher later.

Mechanism

A watcher workflow ticks every ~20 minutes and derives its watch plan from the workflow files themselves (.github/scripts/derive-watch-plan.py). Image digests come from registry manifest HEADs (Docker Hub + ghcr.io; not counted toward Hub pull limits) and are compared against s3:hive-results/image-digests.json. Dispatches use client_source=docker (single env knob to flip to git builds).

Pins all four devnet workflows' matrix concurrency groups to constant prefixes so all runs share the per-client+simulator queue. (The old ${{ github.head_ref || inputs }} groups render the inputs object as the literal Object/empty, so different trigger types never shared a queue and the two quick workflows silently shared each other's.)

With the schedules gone, the now-dead schedule-only paths are removed too: the full workflows' dual prepare steps collapse to one, the quick workflows' || 'git' fallbacks go, and client_source defaults to docker everywhere. generic.yaml also drops its unused workflow_artifact_upload dispatch input (nothing passed it) — it had 11 inputs against GitHub's limit of 10, which broke UI dispatching.