Persisting and reloading the conversation context

[volkanunsal]

I'm trying to build an agent to chat with my books. I'm curious what would be the best approach to persist the conversation and reload it next session while also truncating it in a sliding windows fashion. Have you thought about this usecase?

[volkanunsal]

I saw in one of your presentations you note GenAIScript is a glue, not an agent framework. I'm curious how to bridge the gap from a GenAIScript to something like LlamaIndexTS for a usecase like this. Maybe the web server interface can be configured to have persistent memory.

[pelikhan]

You should be able to use workspace.cache to create a durable set of messages and use a small model with long context like got-4o-mini to keep distillating information into that cache or asking for questions about.

You could also eventually also use RAG (retrieval) if it becomes to too large.

Currently the agent has shared temporary memory that gets queried by the "small" model, but nothing is persisted.

[pelikhan]

Another idea would be to allow provides a message list as part of runprompt. It would be prepended to the convo.