feat(mcp): drive ghost serve UI for query visualization & charting#36
Draft
murrayju wants to merge 24 commits into
Draft
feat(mcp): drive ghost serve UI for query visualization & charting#36murrayju wants to merge 24 commits into
murrayju wants to merge 24 commits into
Conversation
Enable the local MCP server (ghost mcp, stdio mode) to lazily start an
in-process ghost serve webapp and drive it from MCP tools so an agent
can run queries and render charts in a live browser UI that a human can
watch.
- Add an agent bridge (internal/serve/agent.go) that manages SSE clients
with a single active tab, serialized one-at-a-time requests, heartbeat
+ disconnect detection (60s idle), and explicit takeover.
- Expose /api/agent/{events,respond,activate} on the serve handler.
- Lazily start an in-process serve server and open a browser from the
MCP process (internal/mcp/browser.go), torn down on shutdown. Gated to
local/stdio mode only.
- ghost_sql gains limit (default 50, caps all paths), visualize, and
chart_config; add ghost_chart and ghost_ui_state tools.
- Frontend agent layer (web/src/agent/): app-level SSE subscriber,
store-driven query run via the widget apiRef, off-screen ECharts
screenshot, heartbeat, and an active/take-over status banner.
The chart screenshot is rendered off-screen (a detached ECharts instance), so it doesn't depend on which view the user is looking at. Always render and return it so the agent can inspect the data visually: - ghost_sql --visualize: return the chart image for both 'table' and 'chart' views. For 'chart' a render failure is a real error; for 'table' the image is best-effort (failures are swallowed). - ghost_ui_state: return a chart image of the last successful run regardless of the active view (best-effort). Update tool descriptions accordingly.
… run A broken chart_config (or data the config can't plot) no longer fails the whole ghost_sql/ghost_ui_state call. Instead the run data is always returned and a chart_error field explains why no image is present. This applies uniformly to both the table and chart visualize views. - Add chartError to the browser wire types (visualizeResult, uiStateResult) and chart_error to SQLOutput/UIStateOutput. - dispatch.ts: tryRenderChart returns image-or-error instead of throwing. - Surface the chart error in the tool's text summary.
Use the agent SSE stream as a backend-liveness signal. When the stream drops (the `ghost serve`/`ghost mcp` process that served the page is gone), show a banner with the exact `ghost serve` command to restart the backend on the same port — including a copy button. EventSource auto-reconnects, so the banner clears itself once the backend is back. - Serve /api/agent/events as a liveness stream even without an agent bridge (plain `ghost serve`), holding the connection open until the client disconnects. - Track connection lifecycle in the agent store (connecting / connected / disconnected) via EventSource onopen/onerror, distinct from agent presence (status events). Avoids flashing the banner on first load. - serveCommand() builds the restart command from window.location, pinning the current port and adding --host only for non-loopback hosts.
…estart hint Add short flag aliases -p (--port) and -n (--no-open) to `ghost serve`. The disconnected-banner restart command now includes -n, since the tab is already open and reconnects on its own — no need to spawn a browser. Regenerated CLI docs.
ECharts animates the initial render, and getDataURL captures whatever is on the canvas at that instant — so the agent-facing screenshot was often grabbed mid-transition, yielding a partial graph. Force `animation: false` on the option used for the off-screen capture so the first painted frame is the final one. Only affects the screenshot, never the live chart.
Mirror the query history with a chart config history. A toolbar above the config editor has a history button that opens a modal: entries on the left, and on the right the selected config's source (read-only Monaco) over a live preview rendered against the current query results. "Apply config" replaces the current chart config. Persistence trigger: since a config has no discrete "done" event, record whenever an edited config renders successfully against real rows, debounced 1.5s (longer than the live-render and state-persist debounces) so only configs the user dwells on are kept. A baseline ref suppresses the initially-loaded and just-applied configs so they aren't recorded as fresh edits. The store dedups globally (re-rendering/re-applying a config promotes it to the top rather than duplicating), caps at 100 entries, and supports per-entry removal and clear-all. History is global (not per-database), consistent with query history. Also make ChartView's render debounce per-instance instead of module-scoped, so the modal's preview chart and the main chart don't share a single timer and clobber each other's pending applies.
The chart config editor type-checks the config against EChartsOption (via Monaco's TS language service), showing errors as red squiggles. Those errors were only visible to a human — the agent never saw them. Crucially, many type errors (a misspelled option key, a wrong value type, a chart(data) that doesn't return a valid option) don't throw at runtime, so they never surfaced as chart_error either: the chart just rendered wrong or empty with no feedback. Collect those diagnostics headlessly in the browser (diagnostics.ts) by running the same JS language service over a throwaway model, reusing the editor's configureMonacoForCharts types. Return them as chart_diagnostics (line/column/message/severity) from ghost_sql (visualize), ghost_chart, and ghost_ui_state — alongside the rendered image, since they can be present even when the chart renders. The text summary includes a short diagnostics block. Diagnostics collection is best-effort and time-bounded (5s): a slow or unreachable Monaco CDN never stalls or fails the tool call.
…nt bridge Address code-review findings on the MCP serve visualization flow: - Read-only bypass: the visualized query path (browser /api/executeQuery) built its DSN without the read-only GUC, so an MCP server started with read_only: true could still write via ghost_sql visualize. Thread cfg.ReadOnly into connectionStringForService's BuildConnectionString (covers both one-shot and session queries) and surface readOnly via /api/bootstrap with a read-only indicator badge in the UI header. - Row count: visualized results reported the capped read count as the total, hiding truncation. Thread the widget's true total row count through QueryOutcome and use it in handleVisualize/handleUIState instead of data.rows.length. - Agent bridge wedge: a stalled active SSE client (full event buffer) could block Bridge.Request at the dispatch send, since that select didn't watch the idle timer or p.result. Start the timer before dispatch and include timer.C and p.result in the dispatch select so the request always fails fast.
…enshot DOM leak Address code review findings: - browser_format.go: don't synthesize RowsAffected from returned row count; the browser widget has no Postgres command tag, so leave it zero (RowCount conveys returned rows separately) - sql.go: leave RowsAffected untouched when truncating result rows; it reflects the command tag, not the count of rows returned - server.go: Close() now joins both shutdown errors instead of returning early, so docsProxyClient is always closed - screenshot.ts: always remove the off-screen container even if echarts.init() throws
…ools The ghost_chart and ghost_ui_state tools opened the browser-backed UI without first verifying the API client was available. When the user was logged out (or the client couldn't be created), the web app would fail /api/bootstrap before useAgentBridge mounted, so no active client ever connected and the tool failed with a misleading "no browser connected" error after the full ~60s bridge timeout, instead of surfacing the real auth/config error. Match the ghost_sql pattern: call s.app.GetClient() at the start of both handlers and return the auth/config error immediately.
…e store useChartConfigRecorder's debounced record read the current chartConfig from the store when the 1.5s debounce fired, rather than the config that triggered the successful render. If config A rendered and the user then quickly edited it to an invalid config B before the debounce fired, the recorder would capture B \u2014 storing an unrendered/invalid config in chart history. ChartView now passes the exact config it rendered into onRenderSuccess, and the recorder records that captured value instead of re-reading the store.
When an MCP tool's request was abandoned — the caller canceled its context, the request hit the idle timeout, or another tab took over — the bridge cleared the pending request and returned, but the browser had already started executing the query (e.g. a long-running ghost_sql visualization) and received no signal to stop. The query kept running. Add a "cancel" SSE control event carrying the request ID. The bridge now sends it to the dispatched client on context cancellation and idle timeout (only after the command was actually delivered), and to the superseded — but still-connected — client on takeover. The browser's agent bridge tracks the in-flight command and, on a matching cancel, calls the executor's new cancelQuery() to abort the widget's run. A canceled run completes with status 'canceled' (neither rowsAffected nor error); QueryPanel now resolves the pending agent run on that branch too (reported as failed, "the query was canceled") so the dispatcher's runQuery promise and its heartbeat don't leak.
Address three code-review findings on the MCP visualization bridge: - Scope the agent's last-run state to its database. AgentLastRun now carries databaseId; ghost_chart and ghost_ui_state ignore a run that belongs to a different database than the currently-mounted executor, so they never read or chart results through the wrong panel after a database switch. Collapse the redundant getLastRunId/getLastRun deps into a single getLastRun() returning the full run. - Reject pending agent runs on QueryPanel teardown. A useEffect cleanup keyed on databaseId cancels the in-flight widget query and settles all pendingRuns as failed on unmount/database change, so the dispatcher's runQuery promise (and its heartbeat) can't hang the MCP tool call indefinitely. - Document the chart-config execution trade-off. Note in buildChartOption that an agent-supplied chart_config may now be evaluated via new Function; this is an accepted localhost-only trade-off since the model can already issue arbitrary SQL.
loadClient now returns the config alongside the client and project ID, all from the same app.Load snapshot. Previously the bootstrap and connection-string paths called loadClient and then app.GetConfig() separately, which a concurrent reload could interleave — pairing one request's client/project with another snapshot's ReadOnly. That was especially risky in connectionStringForService, where a stale ReadOnly could let a query bypass read-only enforcement.
stringifyCell now returns "NULL" for a nil cell, matching common.ExecuteQuery's server-side query path. Previously a SQL NULL came back as an empty string only when visualize was used, so ghost_sql results depended on the visualization mode and lost the distinction between SQL NULL and an empty string. Adds a test covering NULL and the other cell conversions.
Carry the Postgres command-tag count (rowsAffected) through the browser agent wire types and into the structured ghost_sql / ghost_ui_state output. Previously browserResultSet always built a common.ResultSet with RowsAffected: 0, so a SELECT or DML visualized through the browser reported rows_affected: 0 even when rows were returned or modified — diverging from the server-side path's contract. The widget already reports rowsAffected on a successful run; this threads it through QueryOutcome, the agent store's last run, the visualize/uiState wire results, and browserResultSet. Adds Go and TS tests covering the propagation.
The go-sdk auto-populates CallToolResult.Content with the structured output's
JSON only when the handler leaves Content unset. handleSQLVisualize and
handleUIState set Content themselves (a human-readable summary plus an optional
chart image), which opted out of that auto-population — so the actual result
rows were carried only in StructuredContent.
Per the MCP spec, a tool returning structured content must also return
functionally-equivalent unstructured content so a client can rely on the text
content alone. Clients that read only the text content (e.g. some MCP adapters)
therefore saw just the summary ("200 row(s) returned, showing first 5") with no
data. The non-visualized ghost_sql path was unaffected since it returns a nil
result and lets the SDK fill Content.
Add structuredOutputContent (serializing the output to a JSON TextContent block,
matching the SDK's behavior) and prepend it in both handlers. Adds a test.
The chart error is already carried in the structured output's chart_error field, which is now included in the tool result content (as a JSON text block). Echoing it in the prose summary too duplicated it in the response. Drop the prose line and rely on the structured field.
When the agent passed a database name (instead of its id) to ghost_sql
with visualize, the name flowed straight through to the web UI, which
selects by id and reflects it in the URL. The selector showed no match
("Select a database...") and the URL showed the name (?db=<name>).
Resolve the ref to the canonical database id on the Go side before
dispatching to the browser, since the backend always has the API client
to do so reliably (whereas the frontend's database list may not be
loaded yet). Also normalize a name-based ?db= URL parameter on the
frontend once the database list loads, so the selector matches and the
URL is rewritten to the id.
When a bridge-backed tab disconnected and then reconnected to a plain `ghost serve` (no MCP), the liveness SSE stream reopened but no status event ever followed. setConnected only flipped connectionState, leaving agentPresent/clientId stale, so the UI kept showing "agent active in another tab" with a disabled "Take over" button. setConnected now clears the agent state and lets the next status event restore it. A bridge-backed reconnect re-sends a status event immediately, so the cleared state is transient there.
browserResultSet built a common.ResultSet without CommandTag, so visualized ghost_sql/ghost_ui_state results reported an empty command_tag in their structured output even though the server-side query path fills it and the field is part of the output contract. The widget already exposes the Postgres command on OnQueryComplete; this threads it through QueryOutcome/AgentLastRun, the visualize/uiState wire types, and browserResultSet so visualized results carry the same command_tag as the non-visualized path. The command verb is only carried for successful runs (the widget reports "UNKNOWN" otherwise).
handleChart duplicated the render path with a bare renderChartImage call, so an invalid chart config rejected and bubbled up as a generic "charting failed" tool error, dropping the Monaco editor diagnostics that would help the agent fix the config. handleChart now reuses tryRenderChart (like the ghost_sql visualize path), returning chartError plus diagnostics on a render failure. The Go handler reports that as a normal result describing the failure and diagnostics rather than failing the call \u2014 the config was still applied to the UI.
The agent-facing chart screenshot called getDataURL with a hardcoded backgroundColor of '#ffffff'. ECharts' getDataURL backgroundColor overrides the option's own backgroundColor for the export, so a config that set a dark backgroundColor rendered correctly on-screen but the captured PNG (returned in the MCP response) was forced back to white. Derive the export background from the evaluated option: honor the config's backgroundColor when set (non-empty string or gradient/pattern object), falling back to white only when the config sets none (ECharts paints transparent by default).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Enables the local MCP server (
ghost mcp, stdio mode) to lazily start an in-processghost servewebapp and drive it from MCP tools. An agent can run queries and render charts in a live browser UI that a human can watch — database selection, editor SQL, query runs, chart config, and view state all sync to the visible UI.Note
Draft PR for testing and iteration. Core functionality is working end-to-end; opening this up to test and refine from here.
How it works
internal/serve/agent.go) — manages SSE clients with exactly one active tab, serialized one-at-a-time requests, heartbeat + disconnect detection (60s idle timeout, no fixed query timeout), and explicit takeover. First-to-connect is active; incumbent stays when new tabs connect; on disconnect the most-recently-connected tab is promoted.GET /api/agent/events(SSE),POST /api/agent/respond,POST /api/agent/activate.internal/mcp/browser.go) — lazily starts an in-process serve server (loopback, ephemeral port) and opens a browser on first use; torn down on MCP shutdown. Gated to local/stdio mode only.Tools
ghost_sqlgainslimit(default 50, caps all result paths),visualize(table/chart, runs in the browser to sync the live UI), andchart_config. Returns ≤limit rows + columns, plus a rendered chart image for both views (the screenshot is drawn off-screen, so it's independent of what the user is viewing). If the chart can't be rendered (bad config or unplottable data), the query still succeeds and achart_errorfield explains why — the run is never failed over a chart problem.ghost_chart— reapplies a chart config to the last run and returns a rendered PNG. Never queries.ghost_ui_state— returns selected database, editor SQL, chart config, current view, last-run status/results (capped), and a chart image of the last successful run (regardless of the active view), orchart_errorif it couldn't be rendered.chart_diagnostics— the chart config editor's type/syntax errors (the same red squiggles a human sees, from Monaco's TS language service checking the config againstEChartsOption). These can appear even when the chart renders, because many type errors (a misspelled option key, a wrong value type) don't throw at runtime but still produce a wrong-looking chart. Collected headlessly in the browser (diagnostics.ts), best-effort and time-bounded so a slow/unreachable Monaco CDN never stalls the call.Frontend (
web/src/agent/)useAgentBridgesubscribes to the SSE stream at the app level (survives database-switch remounts), dispatching commands that drive the live store, run queries via the widget's imperativeapiRef, and screenshot charts off-screen via EChartsgetDataURL.QueryPanelregisters anExecutorinto a module-level registry the dispatcher awaits across remounts.AgentStatusBannershows active/inactive state with a "take over" button.Notes for reviewers
chart_errorrather than failing the call.ghost_sql --visualizerejects queryparameters(the browser path doesn't support parameterized queries).Testing
./checkpasses (build, vet, staticcheck, go test).internal/serve/agent_test.go).Code review fixes (latest)
Addressed findings from a code review of the visualization flow:
/api/executeQuery) built its DSN without the read-only GUC, so an MCP server started withread_only: truecould still execute writes viaghost_sqlvisualize. NowconnectionStringForServicethreadscfg.ReadOnlyintoBuildConnectionString(covering both one-shot and session queries), so visualized queries can't bypass read-only mode./api/bootstrapreportsreadOnly, and the UI header shows a "Read-only" lock badge when it's enabled.QueryOutcomeand used inhandleVisualize/handleUIState, so the truncation warning fires correctly.Bridge.Requestat the dispatch send, because thatselectdidn't watch the idle timer orp.result. The timer now starts before dispatch and the dispatchselectincludestimer.Candp.result, so the request always fails fast (idle timeout, takeover/disconnect, or cancellation). Covered byTestBridgeRequestDoesNotWedgeOnFullBuffer.Second review pass
A follow-up review surfaced three more issues, each fixed in its own commit:
chart/ui_stateopened an unusable browser when logged out — unlikeghost_sql, these tools never checked the API client before starting the browser-backed UI. A logged-out user got a misleading "no browser connected" failure after the full ~60s bridge timeout (the web app fails/api/bootstrapbefore the agent bridge mounts) instead of the real auth error. Both handlers now calls.app.GetClient()up front and return the auth/config error immediately.chartConfigfrom the store when its 1.5s timer fired, not the config that actually rendered. Quickly editing to an invalid config before the timer fired stored that invalid config in history.ChartViewnow passes the exact config it rendered intoonRenderSuccess, and the recorder records that captured value."cancel"SSE control event: the bridge sends it to the dispatched client on cancel/timeout (after delivery) and to the superseded client on takeover, and the browser aborts the in-flight run via the newExecutor.cancelQuery(). Acanceledrun completion now also resolves the pending agent run so its promise/heartbeat don't leak. Covered byTestBridgeRequestCancelsBrowserOnContextCancelandTestBridgeSupersedeCancelsBrowser.Third review pass
A further review (gpt-5.5) surfaced three issues on the agent visualization bridge, all fixed:
AgentLastRunstored only the run ID, so after a database switchghost_chart/ghost_ui_statecould read or chart that run through whicheverExecutorwas currently mounted (the new database's panel), returning mismatched data.AgentLastRunnow carriesdatabaseId; both handlers ignore a last run whosedatabaseIddoesn't match the mounted executor —ghost_charterrors with "no completed query run to chart for the current database", andghost_ui_statereports the run metadata but skips the (mismatched) data/chart. Collapsed the redundantgetLastRunId/getLastRundispatch plumbing into a singlegetLastRun()returning the full run.QueryPanelwas torn down mid-query — an agent-initiated run only settled its promise fromhandleQueryComplete, which never fires for an unmounted panel (e.g. the user switches databases while a query is in flight). The dispatcher'srunQuerypromise — and its 15s heartbeat — would then leak forever, so the server-side idle timeout never fired and the tool call hung. AuseEffectcleanup keyed ondatabaseIdnow cancels the in-flight widget query and settles every pending run as failed on unmount/database change.chart_configevaluated vianew Functioncan originate from an AI agent rather than being hand-authored by the operator. Added a comment inbuildChartOptionnoting this is an accepted localhost-only trade-off: the whole stack runs locally under an MCP server/model the operator already trusts and that can already issue arbitrary SQL, so it grants no capability beyond what's already delegated.Database name normalization
When the agent passed a database name (rather than its id) to
ghost_sqlwithvisualize, the name flowed straight through to the web UI, which selects by id and reflects it in the URL. The selector matched no option ("Select a database…") and the URL showed the name (?db=<name>). Fixed on both sides:handleSQLVisualize) resolves the ref to the canonical database id via the API client before dispatching to the browser, since the backend always has the client to do so reliably (whereas the frontend's database list may not be loaded yet). An invalid ref now surfaces as a real API error instead of a broken UI state.app.tsx) normalizes a name-based?db=URL parameter once/api/databasesloads: if the selected value matches no id but does match a name, it's rewritten to the id (updating both the selector and the URL).Chart screenshot background color
A
chart_configthat set a darkbackgroundColorrendered correctly in the live UI, but the screenshot returned in the MCP response had a white background. The off-screen capture inscreenshot.tscalled ECharts'getDataURLwith a hardcodedbackgroundColor: '#ffffff', and that argument overrides the option's ownbackgroundColorfor the export. The export background is now derived from the evaluated option — honoring the config'sbackgroundColorwhen set (non-empty string or gradient/pattern object) and falling back to white only when the config sets none (ECharts paints transparent by default). Covered byweb/src/agent/screenshot.test.ts.