feat: faster startup, crash restart, readiness states, and robustness improvements#41
Merged
Merged
Conversation
… improvements - Parallelize backend detection and API model fetches (9 detectors and 4 fetches, several with 800ms HTTP timeouts, now run concurrently) so startup and refresh no longer freeze the UI. - Replace the unsafe fcntl non-blocking pipe hack with per-child reader threads: portable (fixes serving on Windows), no unsafe, and the log ring buffer grows from 200 to 1000 lines. - Stop servers with SIGTERM and a 2s grace period before SIGKILL. - Install a panic hook that restores the terminal, so a crash can't leave the shell in raw mode. - Drop mouse capture so terminal text selection/copy works again. - Show real readiness: probe /health (llama-server answers 503 while the model loads) and mark servers ◌ loading… / ● ready in the server card and model table. - Detect ports held by other processes before launching, instead of letting the server crash on bind. - 'r' in the server-exit popup relaunches the crashed server with identical settings. - Add a Host field to the serve modal (e.g. 127.0.0.1 vs 0.0.0.0). - Scroll the logs panel with j/k / Ctrl-u/d / g/G when focused. - Parse GGUF headers for the parameter-count label (fills the Params column for files with uninformative names) and trained context length, warning in the serve modal when ctx-size exceeds it. - Add --version/--help CLI flags. - Deduplicate the three identical OpenAI-style model fetchers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The batch of improvements identified while reviewing the codebase after #39.
Responsiveness
r-refresh cost one probe's latency instead of the sum; first frame renders in ~45ms locally.Robustness
fcntl(O_NONBLOCK)— each child's stdout/stderr is read on a dedicated thread feeding an mpsc channel. Removes theunsafeblock, makes serving work on Windows (where the old code would block the UI forever), preserves the\rprogress-line collapsing, and raises the log ring buffer from 200 to 1000 lines. Reader threads are joined on exit/stop so the crash popup always includes the final output.do_servenow bind-tests the port, so a port held by another process (e.g. Ollama) produces a status message instead of an instant crash popup.UX
/healthonce per second until ready: llama-server answers 503 while loading (verified:000 → 503 → 200), other backends' 404 counts as up. Server cards show◌ … loading…→● ready; the table status column matches.rrelaunches the exited server with identical model/backend/settings (the launch preset now travels with the server handle).0.0.0.0↔127.0.0.1), persisted by save-defaults.j/k,Ctrl-u/d,g/Gscroll the logs panel when focused; title shows(scrolled ▲N).Polish
general.size_label,general.architecture, and<arch>.context_lengthfrom the file header (seeking past arrays, so it's a few KB read regardless of model size). Fills the Params column for files with uninformative names (e.g.gemma-3.Q8_0→2.5B) and the serve modal warns when ctx-size exceeds the model's trained context.--version/--helpCLI flags.Testing
cargo fmtapplied.kill -9→ crash popup →rrestart, log scrolling, external port conflict, and clean process teardown on quit.🤖 Generated with Claude Code