Memory usage when offloading

[aafsmarak]

I have a system with 16GB RAM and intel iGPU.
8.8GB is shared GPU memory. Memory usage is higher when offloading to GPU using -ngl. What could be the reason for this?

[NeoZhangJianyu]

-ngl is used to set the layers to load to GPU.
The value is more, the GPU memory usage is more.
If the value is more than max layers, that means whole layers are loaded to GPU. The usage won't be more.

[aafsmarak]

There is only 128MB dedicated GPU memory in my case as its an iGPU.

Memory usage with .\llama-server.exe -m ..\Qwen3-4B-Instruct-2507-Q8_0.gguf -ngl 99 -c 8192:

Memory usage with .\llama-server.exe -m ..\Qwen3-4B-Instruct-2507-Q8_0.gguf -ngl 0 -c 8192:

As you can see RAM usage is higher when offloading to GPU.

[abc-nix]

And if you use --no-mmap ?

[aafsmarak]

With --no-mmap and -ngl 99 RAM usage is definitely lower than without --no-mmap and is around the same as that without GPU offload.
Thanks