Building llama-cpp-python with CUDA support fails due to GLIBC version incompatibility

[ObaOzai]

Hello,

Issue Summary

Building llama-cpp-python with CUDA support fails due to GLIBC version incompatibility

Environment Details

OS: Ubuntu Linux
GPU: NVIDIA (Driver 570.158.01, CUDA 12.8)
Python Environment: Miniconda3 with Python 3.13
PyTorch: 2.8.0.dev20250622+cu128
Compiler: conda’s x86_64-conda-linux-gnu-c++ (GCC 11.2.0)
/home/oba/miniconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/11.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/local/cuda/lib64/libcublasLt.so.12: undefined reference to log2f@GLIBC_2.27' /home/oba/miniconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/11.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: /usr/local/cuda/lib64/libcublasLt.so.12: undefined reference to __cxa_thread_atexit_impl@GLIBC_2.18’
collect2: error: ld returned 1 exit status

Root Cause

The CUDA library libcublasLt.so.12 requires GLIBC symbols:

log2f@GLIBC_2.27 (from GLIBC 2.27+)
__cxa_thread_atexit_impl@GLIBC_2.18 (from GLIBC 2.18+)
But the system apparently has an older GLIBC version that doesn’t provide these symbols.

Build Command Used CMAKE_ARGS=“-DGGML_CUDA=on -DCMAKE_CUDA_ARCHITECTURES=native” pip install -e . --verbose

What’s Been Tried

Using system CUDA instead of conda CUDA - same error
Both conda CUDA libraries and system CUDA libraries show the same GLIBC dependency issue
The error occurs specifically when linking vision tools (llava, mtmd) that depend on CUDA libraries
Questions for Forum

How to resolve GLIBC version conflicts when building llama-cpp-python with CUDA?
Is there a way to use older/compatible CUDA libraries that don’t require GLIBC 2.27+?
Can the build be configured to skip problematic vision components while keeping core CUDA functionality?
Should I use system compiler instead of conda compiler, or create a different conda environment?
Additional Context

The build progresses successfully until the final linking stage for vision tools
Core CUDA libraries (libggml-cuda.so) appear to build successfully
Only fails when linking final executables that use cuBLAS
This gives forum responders all the technical details they need to help diagnose the specific GLIBC/CUDA library compatibility issue.

Regards,

[thedarkbird]

I made a post with a guide on compiling/installing for (Fedora) linux, might be helpful.

[ObaOzai]

Greetings, Pls. post link bellow. Thanks.

[thedarkbird]

Link to my post: #2043

[marybary0000]

Ugh I actually ran into this exact issue a while back while playing around with llama-cpp-python and trying to enable CUDA support. I didn’t realize how tricky the GLIBC version stuff can get on some distros. Did anyone here manage to work around it without upgrading the whole system? I’d love to hear what worked for others.

[ObaOzai]

I just stoped trying to use GPU with llama-cpp-python. Seems some other people have made it work for their env. im not sure how they made it work or if their soultion would fix the issue on my end.

6.14.0-27-generic #27~24.04.1-Ubuntu

[thedarkbird]

Try my solution, should work for any distro since I'm using a Conda env with the right tools: #2043