Skip to content

feat: faster startup, crash restart, readiness states, and robustness improvements#41

Merged
AlexsJones merged 2 commits into
mainfrom
feat/robustness-improvements
Jul 10, 2026
Merged

feat: faster startup, crash restart, readiness states, and robustness improvements#41
AlexsJones merged 2 commits into
mainfrom
feat/robustness-improvements

Conversation

@AlexsJones

Copy link
Copy Markdown
Owner

What

The batch of improvements identified while reviewing the codebase after #39.

Responsiveness

  • Parallel backend detection & model fetches — the 9 backend detectors (7 of which make HTTP probes with 800ms timeouts) and the 4 API model fetches now run on scoped threads. Startup and r-refresh cost one probe's latency instead of the sum; first frame renders in ~45ms locally.

Robustness

  • Reader threads instead of fcntl(O_NONBLOCK) — each child's stdout/stderr is read on a dedicated thread feeding an mpsc channel. Removes the unsafe block, makes serving work on Windows (where the old code would block the UI forever), preserves the \r progress-line collapsing, and raises the log ring buffer from 200 to 1000 lines. Reader threads are joined on exit/stop so the crash popup always includes the final output.
  • Graceful stop — SIGTERM with a 2s grace period before SIGKILL (unix).
  • Panic hook — restores the terminal (raw mode + alternate screen) before printing the panic, so a bug can't leave the shell unusable.
  • External port conflict detectiondo_serve now bind-tests the port, so a port held by another process (e.g. Ollama) produces a status message instead of an instant crash popup.

UX

  • Readiness states — servers probe /health once per second until ready: llama-server answers 503 while loading (verified: 000 → 503 → 200), other backends' 404 counts as up. Server cards show ◌ … loading…● ready; the table status column matches.
  • Restart from crash popupr relaunches the exited server with identical model/backend/settings (the launch preset now travels with the server handle).
  • Host field in the serve modal (switch 0.0.0.0127.0.0.1), persisted by save-defaults.
  • Log scrollbackj/k, Ctrl-u/d, g/G scroll the logs panel when focused; title shows (scrolled ▲N).
  • Mouse capture dropped — it was enabled but unhandled, and it blocked normal terminal text selection/copy.

Polish

  • GGUF header parsing — reads general.size_label, general.architecture, and <arch>.context_length from the file header (seeking past arrays, so it's a few KB read regardless of model size). Fills the Params column for files with uninformative names (e.g. gemma-3.Q8_02.5B) and the serve modal warns when ctx-size exceeds the model's trained context.
  • --version / --help CLI flags.
  • Deduplicated the three identical OpenAI-style model fetchers in backends.rs.

Testing

  • 51 tests pass (3 new GGUF parser tests incl. truncated/corrupt headers), clippy warnings down from 39 to 27 (no new ones), cargo fmt applied.
  • Exercised end-to-end in tmux against real llama-server + model: startup timing, params column, host/context fields, serve → ready transition, external kill -9 → crash popup → r restart, log scrolling, external port conflict, and clean process teardown on quit.

🤖 Generated with Claude Code

@AlexsJones
AlexsJones merged commit 74a5b21 into main Jul 10, 2026
6 checks passed
@AlexsJones
AlexsJones deleted the feat/robustness-improvements branch July 10, 2026 08:57
AlexsJones and others added 2 commits July 10, 2026 10:35
… improvements

- Parallelize backend detection and API model fetches (9 detectors and
  4 fetches, several with 800ms HTTP timeouts, now run concurrently) so
  startup and refresh no longer freeze the UI.
- Replace the unsafe fcntl non-blocking pipe hack with per-child reader
  threads: portable (fixes serving on Windows), no unsafe, and the log
  ring buffer grows from 200 to 1000 lines.
- Stop servers with SIGTERM and a 2s grace period before SIGKILL.
- Install a panic hook that restores the terminal, so a crash can't
  leave the shell in raw mode.
- Drop mouse capture so terminal text selection/copy works again.
- Show real readiness: probe /health (llama-server answers 503 while
  the model loads) and mark servers ◌ loading… / ● ready in the server
  card and model table.
- Detect ports held by other processes before launching, instead of
  letting the server crash on bind.
- 'r' in the server-exit popup relaunches the crashed server with
  identical settings.
- Add a Host field to the serve modal (e.g. 127.0.0.1 vs 0.0.0.0).
- Scroll the logs panel with j/k / Ctrl-u/d / g/G when focused.
- Parse GGUF headers for the parameter-count label (fills the Params
  column for files with uninformative names) and trained context
  length, warning in the serve modal when ctx-size exceeds it.
- Add --version/--help CLI flags.
- Deduplicate the three identical OpenAI-style model fetchers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant