External API: expose Autopilot/CoPilot (conversational + programmatic agent-building)

[goodluck1103]

Summary

Autopilot / CoPilot — the chat-based agent builder in backend/copilot/ — is now the core of the platform, but it has essentially no external API. The full conversational agent-building experience is frontend-locked (internal API only). Enterprises adopting AutoGPT by integrating it into their own products cannot embed the builder, build agents from a natural-language brief programmatically, or drive a chat session over the public API.

This issue tracks exposing Autopilot on the external API surface (/external-api/v1/...). It builds on the hardening in #13556 and the enterprise-readiness epic in #13557 — those cover the plumbing (rate limiting, SDKs, docs, webhooks, idempotency); this covers exposing the core product.

---

Current state (research findings)

What Autopilot is (backend/copilot/): a full stateful chat-based agent builder with 51 tools in copilot/tools/ — e.g. create_agent, edit_agent, customize_agent, fix_agent, validate_agent, run_agent, find_agent, run_block, decompose_goal, agent_generator, agent_browser, e2b_sandbox, manage_schedules, manage_presets.

Served on the INTERNAL API only — /api/chat/* (backend/api/features/chat/routes.py, mounted at backend/api/rest_api.py:426):

• Session lifecycle: POST/GET /sessions, GET/DELETE /sessions/{id}, title/pin/assign-user
• Conversational streaming: POST /sessions/{id}/stream (SSE), /cancel, pending messages
• Usage/paywall: /usage, /usage/reset, config, sharing, suggested-prompts
• Auth = JWT user (frontend), with paywall + credit + rate limiting

What's exposed EXTERNALLY today — only 2 of the 51 tools, statelessly, at /external-api/v1/tools/ (backend/api/external/v1/tools.py):

• POST /tools/find-agent and POST /tools/run-agent, gated by the USE_TOOLS scope.
• These use ephemeral, non-persisted sessions — no conversation, no memory, no building, no streaming. The file header explicitly notes session rate limits don't apply.

Gap: the entire conversational agent-building experience — the platform's core — has no external API. External consumers only get "search marketplace" and "run a marketplace agent."

---

Proposed external API surface

New namespace: /external-api/v1/copilot/*, in three tiers.

Tier 1 — Conversational Autopilot API (embed the builder)

Mirror the internal /api/chat surface for API-key consumers so an enterprise can embed the chat builder in their own product. The SSE streaming machinery already exists internally.

• POST /copilot/sessions — create a session (optionally bound to an existing graph)
• POST /copilot/sessions/{id}/stream — send a message, stream response via SSE
• GET /copilot/sessions/{id} / GET /copilot/sessions/{id}/messages — history
• POST /copilot/sessions/{id}/cancel, DELETE /copilot/sessions/{id}
• New scope: USE_COPILOT (stateful, persisted, billed)

Tier 2 — Programmatic build API (no chat UI needed)

"Give a brief, get an agent" — for CI/CD, internal tooling, agent factories. Long-running → return a job id and pair with completion webhooks (#13557 C-7).

• POST /copilot/agents:build — natural-language spec -> creates/returns a graph (wraps create_agent / decompose_goal / agent_generator)
• POST /copilot/agents/{graph_id}:edit — NL edit instruction (wraps edit_agent / customize_agent / fix_agent)
• POST /copilot/agents/{graph_id}:validate — wraps validate_agent
• New scope: BUILD_AGENT

Tier 3 — Selected tool endpoints (expand beyond the current 2)

Promote more of the 51 tools to stable, documented external endpoints where they stand alone: run-block, list-agent-triggers, manage-schedules, manage-presets, setup-agent-webhook-trigger. Keep sandbox / browser / graphiti tools internal-only (security + cost).

---

Cross-cutting requirements (shared with #13556 / #13557)

• Add USE_COPILOT, BUILD_AGENT to the APIKeyPermission enum (schema.prisma).
• API-key quotas / rate limiting — the external tools file already flags that session limits don't apply (External API: rate limiting, key usage tracking, error mapping & hardening #13556 item 1).
• Async + completion webhooks + idempotency keys for builds/runs (External API: enterprise integration readiness (docs, SDKs, self-service OAuth, webhooks, idempotency) #13557 C-7 / C-8).
• SDKs + streaming docs/examples — an embeddable chat API needs a distributed SDK (External API: enterprise integration readiness (docs, SDKs, self-service OAuth, webhooks, idempotency) #13557 Track A/B).

---

Why this matters / sequencing

Exposing even Tier 1 (embeddable conversational API) or Tier 2 (programmatic build) is the single biggest enterprise-integration lever — it exposes the core product rather than the plumbing around it. Recommended order: Tier 2 (simpler, request/response + webhook) or Tier 1 (higher value, needs SSE + session auth) first; Tier 3 incrementally.

Related

• Security / robustness hardening: External API: rate limiting, key usage tracking, error mapping & hardening #13556
• Enterprise integration readiness epic: External API: enterprise integration readiness (docs, SDKs, self-service OAuth, webhooks, idempotency) #13557