[C++][Gandiva] Upgrade LLVM to 21

[lriggs]

Describe the enhancement requested

Describe the enhancement requested

My company's product uses Gandiva and I recently ran into a bug involving Gandiva, the ORC v2/LLJIT compiler and LLVM versions. I was not able to write a unit test to reproduce the bug outside of our code, though I believe there is a high potential for it to reproduce. It is a memory leak, eventually leading to a crash after running several thousand queries. It only occurred on x86 architectures.

One caveat is that LLVM 21 has not been updated in VCPKG yet. There is a draft PR but it is possible to do the upgrade in Gandiva now since the underlying LLVM packages have been published to the VCPKG servers. I have tested this locally using a VCPKG overlay in my local arrow repo. Here is a draft PR showing how it can be done.

Thank you for your consideration!

Component(s)

C++
Gandiva

Component(s)

C++