[Issue]: wrong O/S library installed into /opt/rocm

[ggankhuy]

Problem Description

Once build as package, I see two different packages: rpm and debian with different size of libhsakmt.so.
This caused, elusive build issue with RHEL/centos kfdtest build because, DEB version was copied into /opt/rocm folder causing link failures.

There is also another issues that libhsakmt.a not libhsakmt.so build with DBUILD_SHARED_LIB=ON with libhsakmt as part of ROCR. In the original ROCT-Thunk (now defunct) I see libhsakmt.so is being built. (Filed this and already solved: #301), Before even attempt to reproduce this issue, the DBUILD_SHARED_LIB shd be fixed since I investigated only .so version not .a version.

(base) [nonroot@localhost roct-thunk]$ ls -l find . -name libhsakmt.so
lrwxrwxrwx. 1 root root 14 Mar 3 15:33 ./ROCT-Thunk-Interface/build/_CPack_Packages/Linux/DEB/hsakmt-6.2.4.99999-Linux/devel/opt/rocm/lib64/libhsakmt.so -> libhsakmt.so.1
lrwxrwxrwx. 1 root root 14 Mar 3 15:33 ./ROCT-Thunk-Interface/build/_CPack_Packages/Linux/RPM/hsakmt-6.2.4.99999-Linux/devel/opt/rocm/lib64/libhsakmt.so -> libhsakmt.so.1
lrwxrwxrwx. 1 root root 14 Mar 3 15:33 ./ROCT-Thunk-Interface/build/libhsakmt.so -> libhsakmt.so.1
(base) [nonroot@localhost roct-thunk]$ ls -l find . -name libhsakmt.so.*
lrwxrwxrwx. 1 root root 18 Mar 3 15:33 ./ROCT-Thunk-Interface/build/_CPack_Packages/Linux/DEB/hsakmt-6.2.4.99999-Linux/devel/opt/rocm/lib64/libhsakmt.so.1 -> libhsakmt.so.1.0.6
-rwxr-xr-x. 1 root root 445816 Mar 3 15:33 ./ROCT-Thunk-Interface/build/_CPack_Packages/Linux/DEB/hsakmt-6.2.4.99999-Linux/devel/opt/rocm/lib64/libhsakmt.so.1.0.6
lrwxrwxrwx. 1 root root 18 Mar 3 15:33 ./ROCT-Thunk-Interface/build/_CPack_Packages/Linux/RPM/hsakmt-6.2.4.99999-Linux/devel/opt/rocm/lib64/libhsakmt.so.1 -> libhsakmt.so.1.0.6
-rwxr-xr-x. 1 root root 251592 Mar 3 15:33 ./ROCT-Thunk-Interface/build/_CPack_Packages/Linux/RPM/hsakmt-6.2.4.99999-Linux/devel/opt/rocm/lib64/libhsakmt.so.1.0.6
lrwxrwxrwx. 1 root root 18 Mar 3 15:33 ./ROCT-Thunk-Interface/build/libhsakmt.so.1 -> libhsakmt.so.1.0

now If i dont build as a package and issue make install, it will silently copy the debian version onto /opt under centos/rhel, whch I only found by comparing the sizes under package build option:

(base) [nonroot@localhost build]$ ls -l sudo find /opt -name "libhsakmt.so*"
lrwxrwxrwx. 1 root root 14 Mar 3 15:38 /opt/rocm-6.3.0/lib64/libhsakmt.so -> libhsakmt.so.1
lrwxrwxrwx. 1 root root 18 Mar 3 15:38 /opt/rocm-6.3.0/lib64/libhsakmt.so.1 -> libhsakmt.so.1.0.6
-rwxr-xr-x. 1 root root 445816 Mar 3 15:38 /opt/rocm-6.3.0/lib64/libhsakmt.so.1.0.6
(base) [nonroot@localhost build]$ cat /etc/os-release
NAME="CentOS Stream"
VERSION="9"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="9"
PLATFORM_ID="platform:el9"
PRETTY_NAME="CentOS Stream 9"

Operating System

CENTOS9/RHEL9

CPU

EPYC

GPU

mi250

ROCm Version

rocm6.2

ROCm Component

ROCR-Runtime

Steps to Reproduce

TBD

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support

No response

Additional Information

No response

[ggankhuy]

Here are the patches I made to get it working under ROCR (under function ROCR_Runtime()) to get working:

1. patched add_library
2. change LIBHSAKMT_PATH to /opt/rocm/lib (from /opt/rocm/ as defunct repo's CMakeLists.txt changed converseily the LIBHSAKMT_PATH)
3. copy RPM version to /opt/rocm

https://gitlab.com/codelabs8265339/codelab-scripts/-/blob/master/build-install-scripts/rocm/sh/build.sh?ref_type=heads#L559

[systems-assistant]

This issue has been migrated to: ROCm/rocm-systems#291