Skip to content

Commit 7100f86

Browse files
affandaraffandar
committed
docs(proposals): token ledger — user-facing capability guide
The same system described as capabilities rather than design, for users and admins, with a capability matrix up front. Written while the design is still a proposal on purpose: a capability that reads badly as documentation is much cheaper to change now than after it ships. Covers what a user can do without ever thinking about pools (the answer is mostly "carry on as today"), what changes quietly in the model picker, how a pause reads and why your own meter can say 28% while you are stopped, and what an admin controls — sharing, limits, the overcommit choice and its consequence, and the three ways to unblock someone. Marked clearly as describing a system that is not built yet.
1 parent 63412be commit 7100f86

2 files changed

Lines changed: 279 additions & 0 deletions

File tree

‎docs/proposals/token-ledger-user-guide.md‎

Lines changed: 276 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,276 @@
1+
# Token pools — what you can do
2+
3+
Status: DRAFT, written against the design in `token-ledger.md`. **Nothing here
4+
is built yet.** This is the user-facing view of the proposal, written early on
5+
purpose: if a capability reads badly here, it is easier to change now than
6+
after it ships.
7+
8+
Audience: everyone who uses the cluster, plus the admins who run it.
9+
10+
## The idea in one paragraph
11+
12+
Every session spends tokens from a **pool**. A pool has a budget — say 10
13+
million tokens a day — and when the budget runs out, sessions drawing on that
14+
pool pause until the window resets, then carry on where they left off. Pools
15+
nest: yours sits inside a bigger one your admin controls. You can see what you
16+
have, what you have spent, and what stopped you.
17+
18+
## What you can do
19+
20+
| | Anyone | Pool owner | Admin |
21+
|---|:--:|:--:|:--:|
22+
| See the pools you can spend from | ● | ● | ● |
23+
| See your own consumption and what is left | ● | ● | ● |
24+
| Set your default pool | ● | ● | ● |
25+
| Choose a different pool for one session | ● | ● | ● |
26+
| Add your own provider key and get a private pool | ● | ● | ● |
27+
| Pause your own session to save budget | ● | ● | ● |
28+
| Create a pool inside one you already have | | ● | ● |
29+
| Split your allocation between sub-pools | | ● | ● |
30+
| Set limits *inside* your own subtree | | ● | ● |
31+
| Share a pool with named users | | ● | ● |
32+
| Open a pool to everyone | | | ● |
33+
| Set limits on shared pools | | | ● |
34+
| Set the cluster default pool and new-user policy | | | ● |
35+
| See fleet-wide consumption and every paused session | | | ● |
36+
| Run a session with no pool at all | | | ● |
37+
| Add or retire providers and models | | | ● |
38+
39+
"Pool owner" means any pool you created or were given admin on — including the
40+
private pool you get from your own API key.
41+
42+
---
43+
44+
# For users
45+
46+
## Finding your pools
47+
48+
Open the **Usage** section of the console. You see only the pools you can
49+
actually spend from: ones shared with you, ones open to everyone, ones you
50+
created, and the private pool behind your own API key if you added one.
51+
52+
Each pool shows its friendly name, what it is rooted at, and a meter for every
53+
limit on it:
54+
55+
```
56+
Azure OpenAI · shared day · all models [####······] 42% 4.2M / 10M resets 00:00Z
57+
week · all models [#########·] 95% 47.5M / 50M resets Mon 00:00Z
58+
Crawlers day · all models [#####·····] 51% 3.1M / 6M resets 00:00Z
59+
```
60+
61+
A pool can carry more than one limit. Read them all — the one nearest full is
62+
the one that will stop you, and it is often not the one you were watching.
63+
64+
## Picking where a session spends
65+
66+
You do not have to. Creating a session works exactly as it does today: pick a
67+
provider and model, then reasoning, context and agent.
68+
69+
Two things change quietly:
70+
71+
- **You only see providers you can pay for.** If you have no pool on GitHub
72+
Copilot, Copilot models are not offered. The picker stops showing you things
73+
that would fail.
74+
- **A "Pool" tab appears** in the same dialog. Ignore it and your default is
75+
used. Open it when you have more than one pool on that provider and care
76+
which budget pays — a crawler budget rather than your interactive one.
77+
78+
Set your default once in the Usage section and most days you never think about
79+
this again.
80+
81+
One rule worth knowing: **a pool can only pay for its own provider.** If your
82+
default is an Azure OpenAI pool and you pick a Copilot model, your default
83+
cannot pay, so the next candidate is used instead. Nothing fails; the session
84+
just spends somewhere sensible.
85+
86+
## Bringing your own key
87+
88+
Add your own provider key — a GitHub Copilot key, say — and you get **your own
89+
provider and your own private pool**. Usage on it counts against you, not the
90+
shared cluster budget, and nobody else can see or spend it.
91+
92+
Your admin may put a safety cap on it. That is there to stop a runaway crawler
93+
spending your personal allowance overnight, not to limit you day to day.
94+
95+
## Splitting your own allowance
96+
97+
Whatever you are given, you can subdivide. A common shape:
98+
99+
```
100+
Your pool 10M / day (given to you by an admin)
101+
├── crawlers 6M / day (you set this)
102+
└── interactive (no limit of its own — takes what is left)
103+
```
104+
105+
You cannot give yourself more than you were given. A sub-pool can narrow your
106+
allowance, never widen it. What it buys you is protection from yourself: a
107+
crawler that goes haywire runs out of *its* 6M and stops, and your interactive
108+
work keeps going.
109+
110+
## When you run low
111+
112+
Three stages, and you see all of them.
113+
114+
**At 70%** — configurable, so your cluster may differ — the session tells you it
115+
is entering degraded mode. The agent is asked to trim what it can: stretch out
116+
recurring jobs, drop optional work. Nothing stops.
117+
118+
**At 100%** the session pauses. It does not fail and it does not lose its place.
119+
The session shows as paused-by-quota with the reason, and it wakes automatically
120+
when the window resets.
121+
122+
**On reset** it resumes on its own. No action needed.
123+
124+
If several limits apply, the one that stops you is named explicitly — including
125+
when it belongs to a pool further up the tree that you do not own. That is why
126+
your own meter can read 28% while you are stopped: someone else spent the shared
127+
budget you both draw on.
128+
129+
## Reading a pause
130+
131+
```
132+
Paused — quota exceeded
133+
Pool Azure OpenAI · shared (an ancestor of your pool)
134+
Limit day · all models · 100M tokens
135+
Used 100M of 100M
136+
Resumes 2026-08-16 00:00 UTC (in 9h 42m)
137+
```
138+
139+
The pool line is the important one. When it names a pool you do not own, your
140+
own budget is fine and you are waiting on a shared one. Ask an admin, or wait
141+
for the reset.
142+
143+
## Pausing on purpose
144+
145+
You can stop a session from spending without ending it — useful when you want to
146+
keep a budget for something else later. Pause it for a set time or indefinitely,
147+
and resume when you want. It takes effect at the next turn boundary, so an
148+
in-flight turn finishes rather than being cut off.
149+
150+
---
151+
152+
# For admins
153+
154+
## Creating and sharing pools
155+
156+
Create a pool anywhere in the tree and open it up two ways:
157+
158+
- **To named users** — a specific list.
159+
- **To everyone** — one grant that covers current and future users, so a team
160+
pool does not need maintaining as people join.
161+
162+
Every pool gets a friendly name. Use one people will recognise in a picker:
163+
"Research team", "Crawlers", "Azure OpenAI · shared".
164+
165+
## Setting limits
166+
167+
A limit is a pool, a window, and a number of tokens. Optionally one model.
168+
169+
```
170+
Research team day · all models 40M
171+
Research team week · all models 150M
172+
Research team day · gpt-5.6-max 8M
173+
```
174+
175+
All three are checked on every turn and all must pass. Per-model limits mean you
176+
do not need a separate pool just to cap an expensive model.
177+
178+
Windows are `day`, `week` and `month`, aligned to UTC. You can shift the
179+
boundary when the default is wrong — a week that turns over on Wednesday, or a
180+
billing month starting on the 12th.
181+
182+
**Each limit resets on its own schedule.** A daily limit turning over does not
183+
reset a weekly one, and a parent reset does not reset a child. Each limit means
184+
exactly what it says.
185+
186+
## Allocating: overcommit is allowed, deliberately
187+
188+
A child pool may be given more than its parent has. That is how you choose your
189+
policy without any extra machinery:
190+
191+
- **Reservations** — give each of five users 20% of a 100M pool. Everyone is
192+
guaranteed their slice, and unused headroom goes to waste.
193+
- **Overcommit** — give all five users the full 100M. Whoever gets there first
194+
uses it, and a heavy user can starve the others.
195+
196+
Neither is wrong. Overcommit is usually what people want, but be aware of the
197+
consequence: a user can be stopped while their own meter reads 10%, because a
198+
pool above them ran dry. The refusal names which pool, so support that with an
199+
answer rather than a shrug.
200+
201+
## The cluster defaults
202+
203+
Two settings decide what happens to people who never touch any of this:
204+
205+
- **The default pool** — what a session spends from when the user has expressed
206+
no preference.
207+
- **The new-user policy** — whether a new user simply draws on the shared
208+
provider pool, or gets a sub-pool of their own with a set share.
209+
210+
## Watching consumption
211+
212+
The Usage section gives you the fleet: totals for the window, the pool tree with
213+
live meters, top consumers by agent and by user, and — first, because it is what
214+
you want during an incident — **every session currently paused by quota, with
215+
the limit that stopped it and when it clears.**
216+
217+
You can also just ask the **Token Manager** agent. It reports across pools,
218+
forecasts when a pool will run dry at the current burn ("this runs out
219+
Thursday" beats "78%"), explains why a specific session is stopped, and proposes
220+
reallocations for you to approve. It will not change an enforcing limit on a
221+
shared pool by itself.
222+
223+
## Unblocking someone
224+
225+
Three moves, in increasing order of blast radius:
226+
227+
1. **Raise the limit** on the pool that bound them.
228+
2. **Move their session** to a pool with headroom.
229+
3. **Run it unpooled** — admin only, bypasses enforcement entirely. Usage is
230+
still recorded, so the ledger stays complete; it is simply not stopped.
231+
232+
You cannot cancel an exhausted window by clearing a hold. Releasing a hold only
233+
undoes a *manual* pause. A budget that is spent stays spent until it resets.
234+
235+
## Providers and models
236+
237+
You control which providers exist and which models each offers. Providers are
238+
either **shared** — the cluster's own credentials, one pool everyone draws on —
239+
or **per-user**, where each person brings a key and gets a private pool.
240+
241+
Retiring a model marks it deprecated rather than deleting it, so historical
242+
usage stays readable. If you name a successor, sessions on the retired model
243+
move across automatically and record the switch.
244+
245+
## A note on the machinery
246+
247+
System agents — the sweeper, the facts manager, the Token Manager itself — draw
248+
on system pools that **warn but never pause**. That is deliberate. A system pool
249+
that could run dry would stop the very agent whose job is to tell you why
250+
everything stopped.
251+
252+
---
253+
254+
## Questions we expect
255+
256+
**Why did my session stop when my meter says 28%?**
257+
A pool above yours ran out. The pause message names it. Your own budget is fine.
258+
259+
**Can I get more budget by making a new pool?**
260+
No. Any pool you create sits inside one you already have, so it can only divide
261+
what you were given, never add to it.
262+
263+
**Does a paused session lose its work?**
264+
No. It stops between turns and resumes at the reset with its context intact.
265+
266+
**Do my own API key's tokens count against the shared budget?**
267+
No. Your key is your own provider with its own private pool.
268+
269+
**What happens if I go over in a single turn?**
270+
It is allowed. Limits are checked between turns, so a turn already running
271+
finishes even if it goes past the limit. The overspend comes out of the rest of
272+
that window.
273+
274+
**Who can see my usage?**
275+
Admins see fleet-wide consumption. Other users see pools they share with you,
276+
not your private ones.

‎docs/proposals/token-ledger.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -815,6 +815,9 @@ after E would-have-blocked log is empty of surprises for a full week
815815

816816
## Related
817817

818+
- `token-ledger-user-guide.md` — the same system described as capabilities, for
819+
users and admins. Written early on purpose: a capability that reads badly
820+
there is cheaper to change now than after it ships.
818821
- `runtime-metrics.md` — the OTel metrics pipeline (stage 6). Its token counters
819822
are labelled `agent_id` + `model` only; they need `provider` and the corrected
820823
identity from stage 1 before they mean anything.

0 commit comments

Comments
 (0)