Compile llama.cpp with CUDA 12

[artyomboyko]

I am building llamacpp with CUDA 12 support (RTX5000). How can I add support for RTX4000 and RXT5000 using cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=“86;89”? The range of acceptable values for the DCMAKE_CUDA_ARCHITECTURES parameter is not specified in documentation. Should I specify CUDA 12 as 120 or as 12? For example:

cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=”86;89;120”

[0cc4m]

Those are cuda compute capabilities, basically device architectures, not cuda versions. See https://developer.nvidia.com/cuda-gpus

[artyomboyko]

@0cc4m OK. How do I set this parameter correctly so that RTX4000 and RTX5000 can be used?