test(supervisor): cover restart backoff timing - #1
Closed
vernonstinebaker wants to merge 87 commits into
Closed
Conversation
Custom providers (those with a base_url) now bypass the nullclaw probe on create and update. The probe is designed for known providers and can misclassify valid HTTP 200 responses from arbitrary OpenAI-compatible endpoints (e.g. when the endpoint returns an empty completion body). Standard providers (no base_url) continue to use the nullclaw probe as before. The re-validate endpoint returns a clear message for custom providers explaining that /models-based validation is coming in a follow-up. Removes the skip_validation workaround that was auto-retried on the frontend — the fix is now in the backend dispatch logic. Adds base_url and provider_name fields to the provider CRUD API, state, and UI (add form, edit form, card display, wizard step, config editor). Adds 'OpenAI Compatible (custom endpoint)' to the provider picker. Validation: - zig build test -Dbuild-ui=false --summary all: all tests pass, 0 leaks - Manual test against local llmserverplus (http://127.0.0.1:5801/v1): custom provider saves without probe, update skips re-probe, re-validate returns expected not-available message Notes: - Credential validation for custom endpoints via /models endpoint is tracked separately (PR 2)
- GET /api/providers/probe-models?base_url=...&api_key=... fetches available model IDs from any OpenAI-compatible /models endpoint - handleCreate, handleUpdate, handleValidate now probe custom providers via /models instead of silently skipping; create/update are non-blocking (always save, record validation result); validate returns live status - Frontend: Fetch Models button in add/edit forms for custom providers; clickable model chips populate the model field - 9 new backend tests (buildModelsUrl, parseModelIdsJson, isProbeModelsPath, handleProbeModels, handleValidate behaviour, handleCreate timestamp)
…base_url - Replace all real provider names/URLs/ports in test blocks with generic values (custom-llm, https://example.com/v1, sk-test-key, port 19999) - Fix State.load() silently dropping base_url on round-trip: the struct literal in the saved_providers loop was missing .base_url = owned_base_url - Remove dangling orphan load-block left by a partial test edit - UI, syncProviderToInstances, and /models probe changes from prior work included in this commit (were already staged but uncommitted)
- Add ui/src/lib/providers.ts as shared source of truth for provider constants (PROVIDER_OPTIONS, OPENAI_COMPATIBLE_VALUE, LOCAL_PROVIDERS, KNOWN_PROVIDER_VALUES, mergeWithManifestOptions) - providers/+page.svelte and ProviderList.svelte now import from the shared module; openai-compatible always visible in wizard dropdown - orchestrator: extractCustomProvider strips custom provider fields before --from-json call so the binary sees no unknown provider name; patchProviderIntoConfig injects credentials into generated config after binary writes it (models.providers.<name>)
…in --from-json payload The wizard sends both a top-level provider field and a providers array; clearing them to empty string still caused 'UNKNOWN PROVIDER' because nullclaw validates the name in both locations. Fix: replace custom provider entries with 'openai' (a known valid name) so the binary generates a valid base config, then patchProviderIntoConfig writes the real credentials into models.providers.<custom-name> afterwards. Also adds debug prints to confirm the detection path fires.
…m provider install - Add `model` field to CustomProvider; extract from wizard answers before stripping - patchProviderIntoConfig now sets agents.defaults.model.primary to <provider>/<model> - Remove the injected `openai` placeholder key from models.providers after binary runs
… install Custom providers backed by a local LLM may expose models whose vision probe never returns an error response — instead it hangs indefinitely, which blocks the nullclaw gateway HTTP handler during startup and causes the nullhub supervisor health-check to time out after 30 s. patchProviderIntoConfig now appends the installed model to agent.vision_disabled_models whenever a base_url is present, so the probe is skipped and the instance passes its first health check.
downloader.download() was called unconditionally on every install, even when the binary was already present in ~/.nullhub/bin/. This made every reinstall (e.g. updating provider config on an existing instance) pay the full network round-trip cost. Add the same existence check that stageLocalBinary already uses: open the target path; if it succeeds the binary is cached and the download is skipped entirely.
- ensureModelOptions now calls api.probeProviderModels(base_url, api_key) instead of api.getWizardModels for openai-compatible providers; the wizard list-models path calls nullclaw --list-models which does not support arbitrary base_url endpoints - modelKey now includes base_url so different custom endpoints with the same api_key get separate cache entries - server.zig: log the actual error when State.load() fails before falling back to empty state, preventing silent data loss
- WizardRenderer: openai-compatible providers (with base_url) are now validated via probeProviderModels HTTP probe instead of the nullclaw binary; nullclaw does not recognise the 'openai-compatible' provider name so it always returned live_ok=false, blocking the NEXT button - WizardRenderer: import OPENAI_COMPATIBLE_VALUE from providers.ts - Instance page: suppress provider health hint/warn while instance is not running (status != 'running') — stale 'instance_not_running' result from boot probe no longer shows as a provider error - Instance page: re-fetch provider health automatically when the instance transitions from any state to 'running'
nullclaw-dev-local panics with 'programmer bug caused syscall error: AGAIN' when a2a.enabled is false. The --from-json generator always emits enabled:false for fresh installs, so patchProviderIntoConfig now flips it to true while preserving all other a2a fields (url, name, version, etc.). Two tests cover the fix: one for the false→true mutation, one to confirm an already-true value is left unchanged. Also corrects the vision_disabled_models comment: the field is used at inference time to strip image markers, not to suppress the startup probe.
OpenAI-compatible checks were timing out before healthy providers could respond through the probe path, which misreported working configs as failed.
…atible-providers feat(providers): add OpenAI-compatible provider support with base_url
fix(auth): protect bare api root path
…on-proxy test(orchestration): cover proxy status and method handling
# Conflicts: # src/installer/orchestrator.zig
…ailures test(installer): roll back persisted state on start failure
…ndaries test(service): tighten systemctl status parsing
…lease-compat fix(ui): install chat modules from released archives
…imported-standalone fix(instances): report imported standalone live status
…refresh # Conflicts: # src/api/instances.zig
…nary-refresh fix(instances): refresh staged dev-local binaries
Preserve short-read semantics in the Zig 0.16 compat stream on Windows and keep the integration smoke on the real std.http.Client path. Also keeps backend CI bounded with a timeout so a socket regression fails visibly instead of hanging indefinitely.
…-http-smoke test(integration): add structured HTTP smoke harness
…e-load fix(runtime): load persisted runtime state safely
…te-guard fix(ui): avoid resetting instance page on same route
Owner
Author
|
Opened by mistake while preparing the upstream PR. Tracking the real review at nullclaw#60. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Validation