Skip to content

Update CI to use Docker #1329

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 6, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .github/scripts/install_oneAPI.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/bin/bash
# Copyright (C) 2025 Intel Corporation
# Under the Apache License v2.0 with LLVM Exceptions. See LICENSE.TXT.
# SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception

# install_oneAPI.sh - Script for installing Intel oneAPI from the official repository

apt-get update
apt-get install -y gpg-agent gnupg
wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB | gpg --dearmor -o /usr/share/keyrings/oneapi-archive-keyring.gpg
echo 'deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main' > /etc/apt/sources.list.d/oneAPI.list
apt-get update
apt-get install -y intel-oneapi-ippcp-devel intel-oneapi-ipp-devel intel-oneapi-common-oneapi-vars intel-oneapi-compiler-dpcpp-cpp
1 change: 1 addition & 0 deletions .github/workflows/pr_push.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ concurrency:

permissions:
contents: read
packages: read

jobs:
CodeChecks:
Expand Down
62 changes: 29 additions & 33 deletions .github/workflows/reusable_basic.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ on: workflow_call

permissions:
contents: read
packages: read

env:
BUILD_DIR : "${{github.workspace}}/build"
Expand All @@ -14,6 +15,12 @@ env:

jobs:
ubuntu-build:
runs-on: ubuntu-latest
container:
image: ghcr.io/bb-ur/umf-${{ matrix.os }}:latest
options: --user test_user --cap-add=SYS_NICE --cap-add=SYS_PTRACE
volumes:
- ${{ github.workspace }}:${{ github.workspace }}
strategy:
matrix:
os: ['ubuntu-22.04', 'ubuntu-24.04']
Expand All @@ -27,7 +34,7 @@ jobs:
link_hwloc_statically: ['OFF']
cmake_ver: ['default']
include:
- os: 'ubuntu-22.04'
- os: ubuntu-22.04
build_type: Release
compiler: {c: clang, cxx: clang++}
shared_library: 'OFF'
Expand All @@ -38,7 +45,7 @@ jobs:
link_hwloc_statically: 'OFF'
# check minimum supported cmake version
cmake_ver: '3.14.0'
- os: 'ubuntu-22.04'
- os: ubuntu-22.04
build_type: Release
compiler: {c: gcc, cxx: g++}
shared_library: 'ON'
Expand All @@ -48,7 +55,7 @@ jobs:
disable_hwloc: 'OFF'
link_hwloc_statically: 'OFF'
cmake_ver: '3.28.0'
- os: 'ubuntu-24.04'
- os: ubuntu-24.04
build_type: Debug
compiler: {c: gcc, cxx: g++}
shared_library: 'ON'
Expand All @@ -59,7 +66,7 @@ jobs:
link_hwloc_statically: 'OFF'
cmake_ver: 'default'
# test level_zero_provider='OFF' and cuda_provider='OFF'
- os: 'ubuntu-22.04'
- os: ubuntu-22.04
build_type: Release
compiler: {c: gcc, cxx: g++}
shared_library: 'OFF'
Expand All @@ -70,7 +77,7 @@ jobs:
link_hwloc_statically: 'OFF'
cmake_ver: 'default'
# test icx compiler
- os: 'ubuntu-22.04'
- os: ubuntu-22.04
build_type: Release
compiler: {c: icx, cxx: icpx}
shared_library: 'ON'
Expand All @@ -81,7 +88,7 @@ jobs:
link_hwloc_statically: 'OFF'
cmake_ver: 'default'
# test lld linker
- os: 'ubuntu-24.04'
- os: ubuntu-24.04
build_type: Release
compiler: {c: icx, cxx: icpx}
shared_library: 'ON'
Expand All @@ -92,8 +99,8 @@ jobs:
link_hwloc_statically: 'OFF'
llvm_linker: '-DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=lld" -DCMAKE_MODULE_LINKER_FLAGS="-fuse-ld=lld" -DCMAKE_SHARED_LINKER_FLAGS="-fuse-ld=lld"'
cmake_ver: 'default'
# test without installing TBB
- os: 'ubuntu-22.04'
# test without installing TBB
- os: ubuntu-22.04
build_type: Release
compiler: {c: gcc, cxx: g++}
shared_library: 'ON'
Expand All @@ -103,7 +110,7 @@ jobs:
disable_hwloc: 'OFF'
link_hwloc_statically: 'OFF'
cmake_ver: 'default'
- os: 'ubuntu-22.04'
- os: ubuntu-22.04
build_type: Debug
compiler: {c: gcc, cxx: g++}
shared_library: 'ON'
Expand All @@ -113,7 +120,7 @@ jobs:
disable_hwloc: 'ON'
link_hwloc_statically: 'OFF'
cmake_ver: 'default'
- os: 'ubuntu-22.04'
- os: ubuntu-22.04
build_type: Release
compiler: {c: gcc, cxx: g++}
shared_library: 'ON'
Expand All @@ -124,52 +131,41 @@ jobs:
link_hwloc_statically: 'ON'
cmake_ver: 'default'
name: Basic (${{matrix.os}}, build_type=${{matrix.build_type}}, compilers=${{matrix.compiler.c}}/${{matrix.compiler.cxx}}, shared_library=${{matrix.shared_library}}, level_zero_provider=${{matrix.level_zero_provider}}, cuda_provider=${{matrix.cuda_provider}}, install_tbb=${{matrix.install_tbb}}, disable_hwloc=${{matrix.disable_hwloc}}, link_hwloc_statically=${{matrix.link_hwloc_statically}}, cmake_ver=${{matrix.cmake_ver}})
runs-on: ${{matrix.os}}

steps:
- name: Checkout
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
fetch-depth: 0

- name: Install apt packages
run: |
sudo apt-get update
sudo apt-get install -y clang libnuma-dev lcov

- name: Install cmake (non-default version)
if: matrix.cmake_ver != 'default'
run: |
sudo apt-get remove --purge -y cmake
echo ${USERPASS} | sudo -Sk apt-get remove --purge -y cmake
wget https://github.com/Kitware/CMake/releases/download/v${{matrix.cmake_ver}}/cmake-${{matrix.cmake_ver}}-Linux-x86_64.sh
chmod +x cmake-${{matrix.cmake_ver}}-Linux-x86_64.sh
sudo ./cmake-${{matrix.cmake_ver}}-Linux-x86_64.sh --skip-license --prefix=/usr/local
echo ${USERPASS} | sudo -Sk ./cmake-${{matrix.cmake_ver}}-Linux-x86_64.sh --skip-license --prefix=/usr/local

- name: Install hwloc
if: matrix.disable_hwloc == 'OFF'
- name: Uninstall hwloc
if: matrix.disable_hwloc == 'ON'
run: |
sudo apt-get install -y libhwloc-dev
echo ${USERPASS} | sudo -Sk apt-get remove --purge -y '*hwloc*'
echo ${USERPASS} | sudo -Sk apt-get autoremove -y

- name: Install TBB apt package
if: matrix.install_tbb == 'ON'
- name: Uninstall TBB apt package
if: matrix.install_tbb == 'OFF'
run: |
sudo apt-get install -y libtbb-dev
echo ${USERPASS} | sudo -Sk apt-get remove --purge -y '*tbb*'
echo ${USERPASS} | sudo -Sk apt-get autoremove -y

- name: Install oneAPI basekit
if: matrix.compiler.cxx == 'icpx'
run: |
sudo apt-get install -y gpg-agent wget
wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB | gpg --dearmor | sudo tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" | sudo tee /etc/apt/sources.list.d/oneAPI.list
sudo apt-get update
sudo apt-get install -y intel-oneapi-ippcp-devel intel-oneapi-ipp-devel intel-oneapi-common-oneapi-vars intel-oneapi-compiler-dpcpp-cpp

- name: Install g++-7
if: matrix.compiler.cxx == 'g++-7'
run: sudo apt-get install -y ${{matrix.compiler.cxx}}
echo "${USERPASS}" | sudo -Sk ./.github/scripts/install_oneAPI.sh

- name: Get UMF version
run: |
git config --global --add safe.directory $GITHUB_WORKSPACE
VERSION=$(git describe --tags --abbrev=0 | grep -oP '\d+\.\d+\.\d+')
echo "UMF_VERSION=$VERSION" >> $GITHUB_ENV

Expand Down
118 changes: 80 additions & 38 deletions .github/workflows/reusable_fast.yml
Original file line number Diff line number Diff line change
@@ -1,54 +1,106 @@
# Fast builds
# Fast build linux part is working in dockers, Windows is not
name: FastBuild

on: workflow_call

permissions:
contents: read
packages: read

env:
BUILD_DIR : "${{github.workspace}}/build"
INSTL_DIR : "${{github.workspace}}/install-dir"

jobs:
FastBuild:
env:
VCPKG_PATH: "${{github.workspace}}/build/vcpkg/packages/hwloc_x64-windows;${{github.workspace}}/build/vcpkg/packages/tbb_x64-windows;${{github.workspace}}/build/vcpkg/packages/jemalloc_x64-windows"
ubuntu-build:
runs-on: ubuntu-latest
container:
image: ghcr.io/bb-ur/umf-${{ matrix.os }}:latest
options: --user test_user --cap-add=SYS_NICE --cap-add=SYS_PTRACE
volumes:
- ${{ github.workspace }}:${{ github.workspace }}
strategy:
matrix:
include:
- os: windows-latest
- os: ubuntu-24.04
build_tests: 'ON'
extra_build_options: ' -DUMF_BUILD_BENCHMARKS=ON -DUMF_BUILD_BENCHMARKS_MT=ON'
simple_cmake: 'OFF'
# pure C build (Windows)
- os: windows-latest
# pure C build
- os: ubuntu-24.04
# Tests' building is off for a pure C build
build_tests: 'OFF'
extra_build_options: '-DUMF_BUILD_BENCHMARKS=ON'
simple_cmake: 'OFF'
- os: ubuntu-latest
# simplest CMake on Ubuntu 22.04, 24.04
- os: ubuntu-24.04
build_tests: 'ON'
# Windows doesn't recognize 'CMAKE_BUILD_TYPE', it uses '--config' param in build command
extra_build_options: '-DCMAKE_BUILD_TYPE=Release -DUMF_BUILD_BENCHMARKS=ON -DUMF_BUILD_BENCHMARKS_MT=ON'
simple_cmake: 'OFF'
# pure C build (Linux)
- os: ubuntu-latest
# Windows doesn't recognize 'CMAKE_BUILD_TYPE', it uses '--config' param in build command
# Tests' building is off for a pure C build
build_tests: 'OFF'
extra_build_options: '-DCMAKE_BUILD_TYPE=Release -DUMF_BUILD_BENCHMARKS=ON'
simple_cmake: 'OFF'
# simplest CMake on ubuntu-latest
- os: ubuntu-latest
build_tests: 'ON'
extra_build_options: '-DCMAKE_BUILD_TYPE=Release'
simple_cmake: 'ON'
# simplest CMake ubuntu-22.04
- os: ubuntu-22.04
build_tests: 'ON'
extra_build_options: '-DCMAKE_BUILD_TYPE=Release'
simple_cmake: 'ON'
name: Fast builds (${{matrix.os}}, build_tests=${{matrix.build_tests}}, simple_cmake=${{matrix.simple_cmake}}, extra_build_options=${{matrix.extra_build_options}})
runs-on: ${{ (matrix.os == 'ubuntu-latest' && github.repository_owner == 'oneapi-src') && 'intel-ubuntu-22.04' || matrix.os }}
name: Fast (${{matrix.os}}, build_tests=${{matrix.build_tests}}, simple_cmake=${{matrix.simple_cmake}}, extra_build_options=${{matrix.extra_build_options}})

steps:
- name: Checkout repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
fetch-depth: 0

- name: Configure CMake
if: matrix.simple_cmake == 'OFF'
run: >
cmake
-B ${{env.BUILD_DIR}}
-DCMAKE_BUILD_TYPE=Release
-DUMF_FORMAT_CODE_STYLE=OFF
-DUMF_DEVELOPER_MODE=ON
-DUMF_BUILD_LIBUMF_POOL_JEMALLOC=ON
-DUMF_BUILD_TESTS=${{matrix.build_tests}}
-DUMF_BUILD_EXAMPLES=ON
-DUMF_BUILD_LEVEL_ZERO_PROVIDER=ON
-DUMF_BUILD_CUDA_PROVIDER=ON
-DUMF_TESTS_FAIL_ON_SKIP=ON
-DUMF_BUILD_SHARED_LIBRARY=ON
${{matrix.extra_build_options}}

- name: Configure CMake (simple)
if: matrix.simple_cmake == 'ON'
run: >
cmake
-B ${{env.BUILD_DIR}}
-DCMAKE_BUILD_TYPE=Release
-DUMF_BUILD_SHARED_LIBRARY=ON
-DUMF_TESTS_FAIL_ON_SKIP=ON
${{matrix.extra_build_options}}

- name: Build
run: cmake --build ${{env.BUILD_DIR}} --config Release -j $(nproc)

- name: Run examples
working-directory: ${{env.BUILD_DIR}}
run: ctest --output-on-failure --test-dir examples -C Release

- name: Run tests
if: matrix.build_tests == 'ON'
working-directory: ${{env.BUILD_DIR}}
run: ctest --output-on-failure --test-dir test -C Release

# TODO: use dockers
windows-build:
runs-on: windows-latest
env:
VCPKG_PATH: "${{github.workspace}}/build/vcpkg/packages/hwloc_x64-windows;${{github.workspace}}/build/vcpkg/packages/tbb_x64-windows;${{github.workspace}}/build/vcpkg/packages/jemalloc_x64-windows"
strategy:
matrix:
include:
- build_tests: 'ON'
simple_cmake: 'OFF'
- build_tests: 'OFF'
simple_cmake: 'OFF'
- build_tests: 'OFF'
simple_cmake: 'ON'
name: Fast (windows-latest, build_tests=${{matrix.build_tests}}, simple_cmake=${{matrix.simple_cmake}})

steps:
- name: Checkout repository
Expand All @@ -57,24 +109,16 @@ jobs:
fetch-depth: 0

- name: Initialize vcpkg
if: matrix.os == 'windows-latest'
uses: lukka/run-vcpkg@5e0cab206a5ea620130caf672fce3e4a6b5666a1 # v11.5
with:
vcpkgGitCommitId: ea2a964f9303270322cf3f2d51c265ba146c422d # 1.04.2025
vcpkgDirectory: ${{env.BUILD_DIR}}/vcpkg
vcpkgJsonGlob: '**/vcpkg.json'

- name: Install dependencies (windows-latest)
if: matrix.os == 'windows-latest'
- name: Install dependencies
run: vcpkg install --triplet x64-windows
shell: pwsh # Specifies PowerShell as the shell for running the script.

- name: Install dependencies
if: matrix.os != 'windows-latest'
run: |
sudo apt-get update
sudo apt-get install -y cmake libhwloc-dev libnuma-dev libtbb-dev

- name: Configure CMake
if: matrix.simple_cmake == 'OFF'
run: >
Expand All @@ -91,17 +135,16 @@ jobs:
-DUMF_BUILD_CUDA_PROVIDER=ON
-DUMF_TESTS_FAIL_ON_SKIP=ON
-DUMF_BUILD_SHARED_LIBRARY=ON
${{matrix.extra_build_options}}

- name: Configure CMake (simple)
if: matrix.simple_cmake == 'ON'
run: >
cmake
-B ${{env.BUILD_DIR}}
-DCMAKE_INSTALL_PREFIX="${{env.INSTL_DIR}}"
-DCMAKE_PREFIX_PATH="${{env.VCPKG_PATH}}"
-DUMF_BUILD_SHARED_LIBRARY=ON
-DUMF_TESTS_FAIL_ON_SKIP=ON
${{matrix.extra_build_options}}

- name: Build
run: cmake --build ${{env.BUILD_DIR}} --config Release -j
Expand All @@ -118,7 +161,6 @@ jobs:
# TODO: We could add some script to verify metadata of dll's (selected fields, perhaps)
# ref. https://superuser.com/questions/381276/what-are-some-nice-command-line-ways-to-inspect-dll-exe-details
- name: Print metadata of our dll's
if: matrix.os == 'windows-latest'
run: |
get-command ${{github.workspace}}/build/bin/Release/umf.dll | format-list
get-command ${{github.workspace}}/build/src/proxy_lib/Release/umf_proxy.dll | format-list
Loading