feat(tui): add a Jobs section; fix task_size and db_uri passthrough - #29
Merged
Conversation
Adds a Jobs browser to the TUI alongside Tables, plus two silent config-passthrough bugs found while building it. Jobs section - `j` lists recent jobs newest-first across every status, with a status tally on the section header. Selecting one shows the full record: status, target, elapsed, a curated progress line, launch config, every metric, and the complete event log (geneva has no streaming log API, so the record's append-only event list *is* the log). - `f` follows a running job by re-reading the record every 3s, and stops itself once the job reaches a terminal state. - Record reading/formatting moves to core/jobs.py so the `jobs` CLI and the TUI cannot drift; ops/jobs.py sheds ~120 lines. - The mode control drops its `auto` entry: in an interactive app with no command line to record the choice, the control should name the backend you are actually on. fix(backfill): forward task_size on local runs core/backfill.py kept `task_size` in the remote-only branch, on the premise that the local NativeTable API lacks it. In geneva 0.14 `NativeTable = Table` — a plain alias, one backfill implementation — and task_size reaches `run_ray_add_column(task_size=...)` either way, so the setting was silently dropped on every `--mode local` run. Verified with 40 rows: `--task-size 8` and `--task-size 40` both produced 4 tasks before, and now produce 5 and 1. `use_cpu_only_pool` stays remote-only (it requests a CPU_ONLY_NODE Ray resource local Ray never advertises). fix(config): keep the db:// scheme on an enterprise db_uri geneva reads any uri that is not `db://…` as an on-disk database created relative to the working directory, so a scheme-less name silently gave you ./<name>/ instead of the cluster — the source of stray database directories at the repo root. normalize_db_uri() corrects it and logs at WARNING; object-store uris and explicit paths pass through untouched. `--db-uri` now resolves inside load_config() rather than by assignment at five call sites, so no caller can bypass the check. Also: gitignore stray local database directories by their structural markers (the name comes from a gitignored config.yaml, so it cannot be listed), and hide the TUI's db_uri field outside enterprise mode, where geneva ignores it. Coverage goes to 100% line and branch (from 97.06%), 305 tests. Beyond the new code that includes regression guards for both fixes and five pre-existing partial branches in untouched modules. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
justinrmiller
requested review from
BubbleCal,
brendanclement,
jmhsieh,
rpgreen,
sarwarbhuiyan and
wkalt
July 30, 2026 19:15
The comment described the flag as pinning appliers to the CPU-only node pool without noting that it only ever applies to appliers requesting no GPU, which reads as though it could keep a GPU UDF off the GPUs. geneva branches on the UDF's num_gpus first and only reaches the use_cpu_only_pool arm when that is zero (runners/ray/pipeline.py), and the cpu-only resource is advertised only by node groups with num_gpus==0 (runners/ray/raycluster.py). So the flag keeps CPU work from squatting on GPU nodes — it leaves GPU scheduling untouched. Also reword the `lightweight` step's --help text, which is the user-facing surface of the same ambiguity. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The mode/config/db_uri controls only changed what the *next* read would connect to; everything already on screen survived. Switching to enterprise left the local job ids in the nav, the selected job id in `_current_job`, and the follow timer polling — so the next read (a click, `r`, or the 3s poll) asked the cluster for a job that only exists on disk and came back "not found". The controls now define a target key, and moving it retargets the app: the listings are dropped, the selected job/table and the follow poll are cleared, and the section on screen is re-listed against the new backend (jobs if you were reading job records, tables otherwise). Reads carry the epoch they started in, so a slow scan that lands after a switch is discarded instead of repainting the pane with the previous database's rows. Also: - An unusable target no longer takes the app down. `load_config` raises for enterprise-with-no-config.yaml, and that exception propagated out of the event handler into Textual's crash dialog; it now lands as a `⚠` leaf in the nav section, or on the pane the read would have filled. - Refresh/`r` with nothing selected re-lists the section instead of doing nothing — the state a retarget leaves behind. - The header names the backend you are on (`enterprise · db://prod`), and a genuine miss reads `no such job in the local database ./local_db` rather than a bare `not found`. - The table/job load and filter paths funnel through `_open_table` / `_refresh_tables` / `_refresh_jobs` rather than repeating the build-cfg-and-dispatch dance at each call site. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…yaml test_tui_switching_mode_drops_the_other_backends_listings passed locally and failed in CI. Switching the mode Select to enterprise re-lists the browsed section, and enterprise mode requires config.yaml — which is gitignored. On a machine that happens to have one the listing succeeds; in CI (or a fresh clone) load_config raises and the nav gets a `⚠ config file not found` leaf instead of the empty listing asserted. Stub load_config in the three tests that switch backends, so they test retarget semantics rather than the ambient filesystem. Verified from a working directory with no config.yaml: 310 passed, was 1 failed. The production behaviour is unchanged and correct — surfacing the config error into the pane is what should happen when a backend can't be configured. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
sarwarbhuiyan
approved these changes
Jul 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a Jobs section to the TUI alongside Tables — and two silent config-passthrough bugs that surfaced while building it.
Jobs section
jlists recent jobs newest-first across every status, with a tally on the section header (Jobs — 1 RUNNING · 2 DONE). Selecting one shows the full record: status, target column, elapsed, a progress line, the launch config, every metric, and the complete event log — which, since geneva has no streaming log API, is the log.ffollows a running job by re-reading the record every 3s and stops itself once the job goes terminal (the TUI equivalent ofjobs tail).That progress line is curated on purpose: geneva attaches ~35 metrics to a backfill, most of them timers with a zero total.
progress_summary()keeps only the ratio-shaped ones and prefers the three that read as progress. The full set stays in the scrollable detail below.Record reading and formatting moved to
core/jobs.py, so thejobsCLI and the TUI render identically and can't drift;ops/jobs.pysheds ~120 lines. The mode control also drops itsautoentry — in an interactive app there's no command line recording what you picked, so the control should name the backend you're actually on.fix:
task_sizewas dropped on every local runcore/backfill.pykepttask_sizein the remote-only branch, on the premise that the localNativeTableAPI lacks it. That premise is false in geneva 0.14:NativeTable = Tableis a plain alias, there's onebackfillimplementation, andtask_sizereachesrun_ray_add_column(task_size=…)either way.Verified by task count rather than by reading signatures — 40 rows, differing only in
--task-size:--task-size 8--task-size 40Before, both landed on geneva's default sizing; the flag did nothing. This affects every step going through
backfill_columnunder--mode local— the laptop path, where an oversized task on a slow UDF is exactly what trips the stall watchdog.use_cpu_only_poolstays remote-only: it requests aCPU_ONLY_NODERay resource that a local cluster never advertises, so the tasks would sit unschedulable.fix: a scheme-less
db_urisilently became a local databasegeneva reads any uri that isn't
db://…as an on-disk database created relative to the working directory. So a bare--db-uri smokegave you./smoke/while you believed you were on the cluster — the source of the stray database directories that were showing up as untracked files.normalize_db_uri()now restores the scheme and logs it:Object-store uris (
s3://,gs://,az://) and explicit filesystem paths pass through untouched, as does everything in local mode wheredb_uriis unused. The override was previously applied by assigning onto the returnedConfigat five separate call sites, each of which would have bypassed normalization — so--db-urinow resolves insideload_config()and no path can skip the check.Also
.gitignorecovers stray local database directories. Not by name — the name comes fromconfig.yaml, which is itself gitignored — but by the structural markers every geneva database contains (__manifest/,…___system$geneva_jobs/). Verified no currently-tracked file becomes ignored.db_urifield is hidden outside enterprise mode, where geneva ignores it.Testing
Coverage goes to 100% line and branch (from 97.06%), 305 tests passing,
tydiagnostics unchanged at 22.Beyond covering the new code, this adds regression guards for both fixes —
task_sizesurviving the local backfill path,--db-urireaching a generated CLI normalized, and all three ops CLIs forwardingdb_uri_override(parametrized, so it can't silently regress in just one). It also closes five pre-existing partial branches in untouched modules, each a real edge rather than a number: even-division batching, no-jitter retry backoff, theruntime_sessionfallback for when geneva's privateray_clustermoves,skip_null_video=False, and deleting from an empty UDF library.🤖 Generated with Claude Code