Any other mechanisms to save gpu memory other than paged attention?

[lyz-sys]

Hi guys, are there other mechanisms implemented in vllm to save gpu memory other than paged attention? Thank you.

[zhuohan123]

Another technique is continuous batching, which reduces padding memory and computation. You can read this blog to learn more.