Skip to content

Commit dcede10

Browse files
committed
release: PilotSwarm 0.5.18
Ship model-visible image attachments across the portal, TUI, Web API, SDK, and MCP server. Images paste/drop/pick into the portal composer (desktop + mobile), Ctrl+V-paste into the TUI from the OS clipboard, and ride sendMessage as artifact refs; workers fetch bytes, gate on the model's vision capability, and deliver true multimodal blob attachments. Orchestration 1.0.65 (1.0.64 frozen). Also: statusVersion-based stale- status guard (fixes wedged 'Working..'), actionable 4xx artifact errors, 8 MB ingress body limit, configurable worker turn timeout, and the durable-signals design proposal.
1 parent d0ab6d9 commit dcede10

64 files changed

Lines changed: 6865 additions & 564 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.env.example

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@ DATABASE_URL=postgresql://postgres:postgres@localhost:5432/durable_copilot
4949
# PILOTSWARM_FACTS_PG_POOL_MAX=3
5050
# PILOTSWARM_ORCHESTRATION_CONCURRENCY=2
5151
# PILOTSWARM_WORKER_CONCURRENCY=2
52+
# PILOTSWARM_TURN_TIMEOUT_MS=1200000
5253

5354
# ── Optional provider overlays ──────────────────────────────────────────────
5455
# The default facts provider is plain Postgres (PgFactStore) and is configured

CHANGELOG.md

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,66 @@
11
# Changelog
22

3+
## 0.5.18 — 2026-07-21
4+
5+
### Model-visible image attachments across every surface
6+
7+
Operators can now show images to the model: paste (Ctrl/Cmd+V), drag-drop,
8+
or pick images in the portal composer — desktop and mobile — attach from the
9+
TUI with Ctrl+V (OS clipboard) or Ctrl+A (file path), or send attachment refs
10+
through the Web API, SDK, and MCP server. Images persist as session binary
11+
artifacts (upload-first, reference-after); vision-capable models receive them
12+
as true multimodal blob attachments with per-model capability gating. New
13+
sessions use orchestration `1.0.65`.
14+
15+
### SDK
16+
17+
- `sendMessage`/`send` accept `attachments: [{filename}]` refs, validated at
18+
the API edge against the session's artifact store (png/jpeg/gif/webp, 4 MB
19+
per image, 4 images / 8 MB per message) and carried on the durable queue as
20+
resolved refs — bytes never enter orchestration history.
21+
- Orchestration `1.0.65` threads attachment refs through drain, merge, and
22+
continue-as-new (`1.0.64` frozen); the runTurn activity fetches bytes from
23+
the artifact store, gates on the live model catalog's vision capability
24+
(`SessionManager.getModelVisionInfo`), and forwards blob attachments to the
25+
Copilot session. Drops are explicit: prompt omission notes plus
26+
`runtime.attachment_dropped` events.
27+
- Attachment refs ride the `user.message` event for transcript rendering.
28+
29+
### Portal
30+
31+
- Composer image intake: clipboard paste, drag-drop (with drop highlight),
32+
and a paperclip picker (`accept=image/*` offers camera/photo library on
33+
phones); staged images render as thumbnail chips with size and remove.
34+
- Transcript user messages render authenticated thumbnail strips; the Files
35+
pane previews raster image artifacts inline.
36+
- Stale-status guard now trusts the server's monotonic `statusVersion`,
37+
fixing sessions wedged on "Working.." after turn completion.
38+
- Artifact/attachment validation errors return actionable 4xx codes
39+
(`ARTIFACT_CONTENT_TYPE_MISMATCH`, `ARTIFACT_TOO_LARGE`,
40+
`INVALID_ATTACHMENT`) instead of generic 500s.
41+
- Ingress body limit raised to 8 MB for image uploads (deploy manifest).
42+
43+
### TUI
44+
45+
- Ctrl+V pastes an image from the OS clipboard (pngpaste/osascript on macOS,
46+
wl-paste/xclip on Linux, PowerShell on Windows) and stages it on the next
47+
message; Ctrl+A path uploads auto-stage raster images the same way.
48+
49+
### MCP
50+
51+
- `send_message` and `send_and_wait` accept `attachments` (web mode);
52+
`get_capabilities` reports `prompt.imageAttachments`.
53+
54+
### Worker
55+
56+
- Turn timeout is deployment-configurable.
57+
58+
### Docs
59+
60+
- `docs/proposals/image-attachments-in-chat.md` rewritten to match the
61+
shipped architecture; new draft proposal
62+
`docs/proposals/durable-signals-and-webhooks.md` (`wait_for_signal`).
63+
364
## 0.5.17 — 2026-07-20
465

566
### Reliable finite delegation and cleaner shared UI rendering

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
> **Experimental** — This project is under active development and not yet ready for production use. APIs may change without notice.
44
5-
> **Latest release: v0.5.17**Finite delegated work now wakes parents on substantive results, child contracts survive durable session creation, compact GFM table delimiters render correctly, and portal chat cards use the full pane width. New sessions use orchestration `1.0.64`.
5+
> **Latest release: v0.5.18**Model-visible image attachments everywhere: paste, drop, or pick images in the portal (desktop + mobile), Ctrl+V clipboard paste in the TUI, and attachment refs through the Web API, SDK, and MCP server — persisted as session artifacts and vision-gated per model. New sessions use orchestration `1.0.65`.
66
77
A durable execution runtime for [GitHub Copilot SDK](https://github.com/github/copilot-sdk) agents. Crash recovery, durable timers, session dehydration, and multi-node scaling — powered by [duroxide](https://github.com/microsoft/duroxide). Just add a connection string.
88

deploy/envs/template.env

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -169,6 +169,7 @@ INGRESS_CLASS=azure-application-gateway
169169
# own K8s secret keeps the connection-string flow intact.
170170
PILOTSWARM_USE_MANAGED_IDENTITY=1
171171
AZURE_STORAGE_CONTAINER=copilot-sessions
172+
PILOTSWARM_TURN_TIMEOUT_MS=1200000
172173

173174
# ─── Azure AI Foundry (optional) ───
174175
# When true, base-infra provisions a Microsoft.CognitiveServices/accounts

deploy/gitops/worker/overlays/default/.env

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,8 @@ AZURE_TENANT_ID=00000000-0000-0000-0000-000000000000
2525
# custom Azure OpenAI / Anthropic-direct endpoint can add a stamp-local
2626
# kustomize patch that adds them to the worker-env ConfigMap.
2727
AZURE_STORAGE_CONTAINER=copilot-sessions
28+
# Wall-clock cap for one Copilot turn. 0 disables the cap.
29+
PILOTSWARM_TURN_TIMEOUT_MS=1200000
2830
# Storage managed-identity flag + account URL (consumed by
2931
# packages/sdk/src/blob-store.ts → createSessionBlobStore). Setting the
3032
# flag to "1" tells the worker to use DefaultAzureCredential (federated

deploy/k8s/portal-ingress.yaml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,10 @@ metadata:
66
annotations:
77
cert-manager.io/cluster-issuer: "letsencrypt-prod"
88
nginx.ingress.kubernetes.io/ssl-redirect: "true"
9+
# Image attachment uploads are base64 JSON up to ~5.4 MB (4 MB decoded
10+
# cap) on the artifact PUT route; nginx's 1 MB default rejects them with
11+
# 413 before the app's own 8 MB route limit can apply.
12+
nginx.ingress.kubernetes.io/proxy-body-size: "8m"
913
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
1014
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
1115
nginx.ingress.kubernetes.io/proxy-http-version: "1.1"

deploy/k8s/worker-deployment.yaml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,8 @@ spec:
6060
value: "4"
6161
- name: PILOTSWARM_WORKER_CONCURRENCY
6262
value: "8"
63+
- name: PILOTSWARM_TURN_TIMEOUT_MS
64+
value: "1200000"
6365
envFrom:
6466
- secretRef:
6567
name: copilot-runtime-secrets

deploy/scripts/test/foundry-substitute.test.mjs

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ test("__FOUNDRY_ENDPOINT__ in model_providers.json is substituted from FOUNDRY_E
4747
WORKER_IMAGE_TAG: "test",
4848
IMAGE: "acrtest.azurecr.io/pilotswarm-worker:test",
4949
PILOTSWARM_USE_MANAGED_IDENTITY: "1",
50+
PILOTSWARM_TURN_TIMEOUT_MS: "1200000",
5051
PILOTSWARM_CMS_FACTS_DATABASE_URL: "postgres://test/cms",
5152
PILOTSWARM_DB_AAD_USER: "uami",
5253
LOCATION: "westus3",
@@ -99,6 +100,7 @@ test("__FOUNDRY_ENDPOINT__ stays unresolved when FOUNDRY_ENDPOINT is empty/unset
99100
WORKER_IMAGE_TAG: "test",
100101
IMAGE: "acrtest.azurecr.io/pilotswarm-worker:test",
101102
PILOTSWARM_USE_MANAGED_IDENTITY: "1",
103+
PILOTSWARM_TURN_TIMEOUT_MS: "1200000",
102104
PILOTSWARM_CMS_FACTS_DATABASE_URL: "postgres://test/cms",
103105
PILOTSWARM_DB_AAD_USER: "uami",
104106
LOCATION: "westus3",

docs/developer/building/examples/blog-travel-event-scanner-sdk.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -680,7 +680,7 @@ const worker = new PilotSwarmWorker({
680680

681681
// Performance tuning
682682
maxSessionsPerRuntime: 50, // concurrent sessions per worker (default: 50)
683-
turnTimeoutMs: 120_000, // LLM turn timeout (default: 2 minutes)
683+
turnTimeoutMs: 1_200_000, // explicit LLM turn cap (default: 20 minutes)
684684
sessionIdleTimeoutMs: 3_600_000, // idle session cleanup (default: 1 hour)
685685

686686
// Management agents — auto-started system agents for monitoring and maintenance

docs/developer/deploy/aks.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -613,6 +613,7 @@ kubectl create secret generic copilot-runtime-secrets \
613613
--from-literal=PILOTSWARM_FACTS_PG_POOL_MAX="3" \
614614
--from-literal=PILOTSWARM_ORCHESTRATION_CONCURRENCY="2" \
615615
--from-literal=PILOTSWARM_WORKER_CONCURRENCY="2" \
616+
--from-literal=PILOTSWARM_TURN_TIMEOUT_MS="1200000" \
616617
--from-literal=AZURE_STORAGE_CONNECTION_STRING="DefaultEndpointsProtocol=https;AccountName=..." \
617618
--from-literal=AZURE_STORAGE_CONTAINER="copilot-sessions"
618619
```
@@ -624,6 +625,7 @@ Worker sizing is env-driven:
624625
- `PILOTSWARM_FACTS_PG_POOL_MAX` — facts `pg.Pool` max size. Default: `3`.
625626
- `PILOTSWARM_ORCHESTRATION_CONCURRENCY` — Duroxide orchestration concurrency. Default: `2`.
626627
- `PILOTSWARM_WORKER_CONCURRENCY` — Duroxide activity/worker concurrency. Default: `2`.
628+
- `PILOTSWARM_TURN_TIMEOUT_MS` — wall-clock cap for one Copilot turn. Default: `1200000` (20 minutes); `0` disables it.
627629

628630
Provider availability in selectors is env-driven at worker startup. If you add or remove a provider key, refresh the secret and restart the workers; changing the checked-in template alone is not enough, and changing the real `.model_providers.json` only takes effect after the updated file is present in the runtime environment.
629631

@@ -717,6 +719,7 @@ kubectl create secret generic copilot-runtime-secrets \
717719
--from-literal=PILOTSWARM_FACTS_PG_POOL_MAX="3" \
718720
--from-literal=PILOTSWARM_ORCHESTRATION_CONCURRENCY="2" \
719721
--from-literal=PILOTSWARM_WORKER_CONCURRENCY="2" \
722+
--from-literal=PILOTSWARM_TURN_TIMEOUT_MS="1200000" \
720723
--from-literal=AZURE_STORAGE_CONNECTION_STRING="..." \
721724
--from-literal=AZURE_STORAGE_CONTAINER="copilot-sessions" \
722725
--dry-run=client -o yaml | kubectl apply -f -

0 commit comments

Comments
 (0)