`StateAccessor` omits fresh user input from `ConversationState.messages`

[fpoli]

I'm using @openrouter/agent@0.7.0 with a persistent StateAccessor, and I'm trying to understand whether the current persistence behavior is intended.

The docs say StateAccessor persists conversation state so “message history, and tool results survive across runs”, and the ConversationState.messages field is described as the full message history. The multi-run example also says each run appends its input and response to state history.

However, it seems that the fresh input passed to callModel() is not saved into ConversationState.messages. This might be a bug in the SDK.

From reading the built SDK output:

• saveResponseToState() appends response.output
• saveToolResultsToState() appends tool results
• I do not see the fresh caller-supplied input appended to persisted state

So after a process restart, the loaded state can contain assistant/tool output from prior turns, but not the actual user messages that caused those turns.

Note that since the user messages are missing, depending on the model/provider, prompt caching can result in an unintented miss.

Sketch of an example:

const first = await openrouter.callModel({
  model,
  input: "The secred code is 'pizza'. Answer OK.",
  tools,
  state,
}).getText()

console.log("First answer:", first)
// First answer: OK.

const second = await openrouter.callModel({
  model,
  input: "What's the secret code?",
  tools,
  state,
}).getText()

console.log("Second answer:", first)
// Second answer: I don't have a secret code, and I'm not sure what you're referring to. Could you give me more context? I'm happy to help if you can clarify what you're looking for! 

Full example:

import {
  type ConversationState,
  OpenRouter,
  type StateAccessor,
  type Tool,
} from "@openrouter/agent"

let persisted: string | null = null
const state: StateAccessor<readonly Tool[]> = {
  load: async () =>
    persisted
      ? (JSON.parse(persisted) as ConversationState<readonly Tool[]>)
      : null,
  save: async (next) => {
    persisted = JSON.stringify(next)
  },
}

async function main(): Promise<void> {
  const apiKey = process.env.OPENROUTER_API_KEY
  if (!apiKey) throw new Error("Set OPENROUTER_API_KEY")

  const firstClient = new OpenRouter({ apiKey })
  const firstAnswer = await firstClient
    .callModel({
      model: "anthropic/claude-sonnet-4.6",
      input: [
        {
          role: "user",
          content: "The secred code is 'pizza'. Answer OK.",
        },
      ],
      state,
    })
    .getText()
  console.log("First answer:", firstAnswer)

  const secondClient = new OpenRouter({ apiKey })
  const secondAnswer = await secondClient
    .callModel({
      model: "anthropic/claude-sonnet-4.6",
      input: [{ role: "user", content: "What's the secret code?" }],
      state,
    })
    .getText()

  console.log("Second answer:", secondAnswer)

  console.log("\nPersisted messages:")
  console.dir(JSON.parse(persisted ?? "null")?.messages, { depth: null })
}

void main()

[yonixw]

I think your problem is not re-using the client, this works for me:

Code:

import {
  callModel,
  createInitialState,
  OpenRouter,
  tool,
  updateState,
  type ConversationState,
  type StateAccessor,
} from "@openrouter/agent";
import { z } from "zod";
import pkg from "../package.json";

const MODEL = process.env["OPENROUTER_MODEL"] ?? "openai/gpt-5.4-mini";
const APIKEY = process.env["OPENROUTER_API_KEY"] ?? "";

const MODEL_CONFIG = {
  model: MODEL,
  provider: {
    zdr: true, // Zero data retention
    sort: "price",
  },
};

const client = new OpenRouter({
  apiKey: APIKEY,
  appTitle: pkg.name + ": " + pkg.version,
});

const conversations = new Map<string, ConversationState>();
function createStateAccessor(conversationId: string): StateAccessor {
  return {
    load: async () => conversations.get(conversationId) ?? null,
    save: async (state) => {
      conversations.set(conversationId, state);
    },
  };
}

async function simpleState() {
  let state = createStateAccessor("someID");
  state.save(createInitialState("someID"));

  const result = callModel(client, {
    ...MODEL_CONFIG,

    input: [
      {
        role: "user",
        content: "Remember that A =2",
      },
    ],
    state,
  });

  // Get the final text response (tools are auto-executed)
  const text = await result.getText();
  console.log(text);

  const result2 = callModel(client, {
    ...MODEL_CONFIG,
    input: [
      {
        role: "user",
        content: "What is A+3?",
      },
    ],
    state,
  });

  // Get the final text response (tools are auto-executed)
  const text2 = await result2.getText();
  console.log(text2);
}

simpleState();

With this output:

$ bun src/chat.ts
Got it — I’ll remember that **A = 2** for this conversation.
Since **A = 2**, then **A + 3 = 5**.

package.json

{
  "name": "test",
  "version": "0.0.1",
  "private": true,
  "devDependencies": {
    "@types/bun": "latest"
  },
  "peerDependencies": {
    "typescript": "^5"
  },
  "scripts": {
    "start": "bun src/chat.ts"
  },
  "dependencies": {
    "@openrouter/agent": "^0.7.1",
    "@types/node": "^25.9.1",
    "zod": "^4.4.3"
  }
}

[fpoli]

Thanks for the suggestion. My use case is to persist a conversation to be resumed later, in a different process, so I cannot reuse the same client.

Anyway, even your example shows exactly the issue that I described. I believe it's an important bug in the library, but difficult to notice because LLMs are usually verbose enough in their answers to make user inputs almost redundant.

For example, if you change

content: "Remember that A =2",

to

content: "Remember that A =2. Answer only OK.",

then the second LLM answer degrades to something like

Could you clarify what **A** refers to? For example:

- If **A** is a **number**, please tell me its value.
- If **A** is a **variable**, then the answer is simply **A + 3**.
- If you're referring to something specific (like a grade, a code, etc.), please provide more context.

I'm happy to help once I know what **A** represents! 😊

Moreover, if you inspect the content of the state at the end of your simpleState function

console.dir(conversations.get(conversationId)?.messages, { depth: null })

you can see that it does not contain user inputs:

[
  {
    content: [
      {
        annotations: [],
        logprobs: [],
        text: 'OK',
        type: 'output_text'
      }
    ],
    id: 'msg_tmp_5t5ea1glwd7',
    role: 'assistant',
    status: 'completed',
    type: 'message'
  },
  {
    content: [
      {
        annotations: [],
        logprobs: [],
        text: 'Could you clarify what **A** refers to? For example:\n' +
          '\n' +
          '- If **A** is a **number**, please tell me its value.\n' +
          '- If **A** is a **variable**, then the answer is simply **A + 3**.\n' +
          "- If you're referring to something specific (like a grade, a code, etc.), please provide more context.\n" +
          '\n' +
          "I'm happy to help once I know what **A** represents! 😊",
        type: 'output_text'
      }
    ],
    id: 'msg_tmp_xo4h9aed3u9',
    role: 'assistant',
    status: 'completed',
    type: 'message'
  }
]

[yonixw]

I do see user input running

console.dir(conversations.get("someID")?.messages, { depth: null });

For example:

[
  {
+    role: "user",
    content: "Remember that A =2",
  }, {
    content: [
      {
        annotations: [],
        logprobs: [],
        text: "Got it — I’ll remember that **A = 2**.",
        type: "output_text",
      }
    ],
    id: "msg_tmp_mwb4qg1feg",
    phase: "final_answer",
-    role: "assistant",
    status: "completed",
    type: "message",
  }, {
+    role: "user",
    content: "What is A+3?",
  }, {
    content: [
      {
        annotations: [],
        logprobs: [],
        text: "If **A = 2**, then **A + 3 = 5**.",
        type: "output_text",
      }
    ],
    id: "msg_tmp_1152p1jtg6",
    phase: "final_answer",
-    role: "assistant",
    status: "completed",
    type: "message",
  }
]

I think like you said, maybe the separation of client or some specific implementation affect it.

I tried your example and still got 2 roles in the messages, but the AI just gave different answers ("OK", "5"), which is not what we asked in terms of logic, but the state structure/list was not different.

Also, just tried and if the AI calls tools, I also see them there. (type: "function_call" and type: "function_call_output")

[fpoli]

You are right, thanks! This bug has been fixed a few hours after I opened this issue. I was still on version 0.7.0, and I didn't notice that 0.7.1 has been released since then.