Serializing the running model context and Cache-Augmented Generation (CAG)

[RomanKryvolapov]

Hi,
I am working on an Android application that uses several backends, one of them is llama.cpp.

https://github.com/RomanKryvolapov/Local_AI_Launcher

1. I would like to add Cache-Augmented Generation (CAG) to the application, but I have not found a reliable way to do this.
   Simple copying through the buffer takes a lot of time.
   Please tell me, maybe there are ready-made solutions compatible with Java JNI.

2. Is there any future support for NPU?
   Google is currently working on such a project

https://ai.google.dev/edge/litert/next/npu

https://github.com/google-ai-edge/LiteRT-LM