Skip to content

PERF: Add ccache/sccache support to GitHub Actions CI workflows#5910

Open
hjmjohnson wants to merge 3 commits intoInsightSoftwareConsortium:mainfrom
hjmjohnson:add_ccache_build_in_ci
Open

PERF: Add ccache/sccache support to GitHub Actions CI workflows#5910
hjmjohnson wants to merge 3 commits intoInsightSoftwareConsortium:mainfrom
hjmjohnson:add_ccache_build_in_ci

Conversation

@hjmjohnson
Copy link
Member

@hjmjohnson hjmjohnson commented Mar 9, 2026

Summary

  • Add hendrikmuhs/ccache-action@v1.2 to pixi.yml (ubuntu, windows, macos) and arm.yml (ubuntu-arm, macos rosetta, macos Python) to cache compiled objects between CI runs
  • Add configure-ci pixi task in pyproject.toml that passes -DCMAKE_C_COMPILER_LAUNCHER=ccache and -DCMAKE_CXX_COMPILER_LAUNCHER=ccache to CMake; pixi.yml uses this task instead of configure
  • arm.yml injects the same cmake launcher variables via dashboard_cache so the CTest dashboard script picks them up at configure time
  • Cache sized at 11G to accommodate the full ITK object file footprint; keyed on OS + pixi.lock hash (pixi workflow) or OS + matrix name (arm workflow) to avoid stale cache on toolchain upgrades

Test plan

  • Verify CI runs complete successfully on first run (cold cache — expect no speedup)
  • Verify subsequent CI runs show ccache hit rate > 0% in the ccache-action summary
  • Confirm no build failures introduced by the compiler launcher on all three OS platforms in pixi.yml
  • Confirm arm workflow builds succeed with ccache launcher on ubuntu-arm, macos rosetta, and macos Python matrix entries

🤖 Generated with Claude Code

@github-actions github-actions bot added type:Infrastructure Infrastructure/ecosystem related changes, such as CMake or buildbots type:Enhancement Improvement of existing methods or implementation labels Mar 9, 2026
@hjmjohnson
Copy link
Member Author

On my local computer, using ccache GREATLY accelerates the build process when only a single test is modified. Touching the CMakeLists.txt files often triggers the need to rebuild 6760 object files, which takes about 30 minutes; with CCache, it is done in about 2 minutes (with mostly cache hits). I think this will be the case for many CI builds if we can get this to work.

@hjmjohnson hjmjohnson force-pushed the add_ccache_build_in_ci branch from 62410c1 to 04d71ee Compare March 9, 2026 16:55
@hjmjohnson hjmjohnson changed the title ENH: Add ccache support to GitHub Actions CI workflows WIP: DO NOT COMMIT: PERF: Add ccache/sccache support to GitHub Actions CI workflows Mar 9, 2026
@hjmjohnson hjmjohnson force-pushed the add_ccache_build_in_ci branch from 04d71ee to e5285c0 Compare March 9, 2026 17:04
@github-actions github-actions bot added type:Testing Ensure that the purpose of a class is met/the results on a wide set of test cases are correct and removed type:Enhancement Improvement of existing methods or implementation labels Mar 9, 2026
Copy link
Member

@dzenanz dzenanz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@hjmjohnson hjmjohnson force-pushed the add_ccache_build_in_ci branch 5 times, most recently from 7058336 to 4f51f7e Compare March 9, 2026 22:01
@github-actions github-actions bot added the area:Segmentation Issues affecting the Segmentation module label Mar 9, 2026
@hjmjohnson hjmjohnson force-pushed the add_ccache_build_in_ci branch from 4f51f7e to 0fc1f76 Compare March 9, 2026 22:15
@github-actions github-actions bot removed the area:Segmentation Issues affecting the Segmentation module label Mar 9, 2026
@hjmjohnson hjmjohnson changed the title WIP: DO NOT COMMIT: PERF: Add ccache/sccache support to GitHub Actions CI workflows PERF: Add ccache/sccache support to GitHub Actions CI workflows Mar 9, 2026
@hjmjohnson hjmjohnson marked this pull request as ready for review March 10, 2026 00:31
@hjmjohnson hjmjohnson requested review from blowekamp and dzenanz March 10, 2026 00:31
@hjmjohnson hjmjohnson force-pushed the add_ccache_build_in_ci branch from 0fc1f76 to 3ccfa73 Compare March 10, 2026 00:44
@github-actions github-actions bot added the type:Performance Improvement in terms of compilation or execution time label Mar 10, 2026
@hjmjohnson
Copy link
Member Author

NOTE: Compiler object caching with very few changes can reduce the build time to under 10 minutes for builds that previously took 2 hours.

Copy link
Member

@dzenanz dzenanz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds like a great improvement.

hjmjohnson and others added 3 commits March 10, 2026 07:22
Add hendrikmuhs/ccache-action to pixi.yml and arm.yml to cache
compiled objects between CI runs. Add configure-ci pixi task that
sets CMAKE_C_COMPILER_LAUNCHER and CMAKE_CXX_COMPILER_LAUNCHER to
ccache; arm.yml injects the same variables via dashboard_cache.

The BRAINSia/free-disk-space step leaves the apt package index stale,
causing hendrikmuhs/ccache-action to fail installing ccache with exit
code 100. Set update-package-index: true so the action refreshes the
index before attempting the install.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
For each job across all 7 AzurePipelines yml files:
- Add CCACHE_DIR variable pointing to $(Pipeline.Workspace)/.ccache
- Install ccache via apt-get (Linux), brew (macOS), or choco (Windows)
- Add Cache@2 task to restore/save the ccache directory keyed on
  Agent.OS and Build.SourceVersion with OS-only restoreKeys fallback
- Inject CMAKE_C_COMPILER_LAUNCHER and CMAKE_CXX_COMPILER_LAUNCHER
  into dashboard_cache so the CTest dashboard configure step picks
  them up

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Emit STATUS not FATAL_ERROR

When the compiler launcher binary reports a version matching "sccache X.Y.Z"
(e.g. the binary installed by hendrikmuhs/ccache-action on Windows), emit a
CMake STATUS instead of a FATAL_ERROR. sccache works as a compiler launcher
but does not support CCACHE_VERSION reporting.
@hjmjohnson hjmjohnson force-pushed the add_ccache_build_in_ci branch from 3ccfa73 to 1f22093 Compare March 10, 2026 12:23
Copy link
Member

@blowekamp blowekamp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great addition!!! I used to use it with CircleCI with SimpleITK, but never reimplemented in with GitHub Actions.

Please check the documentation for the hash keys, there is some tuning that can be done to better get a good cache.

We have had some disk space issues on several builds. Maybe it was just the Python builds? This may cause problems with them. It may take a couple runs for the cache to get filled, and the disk space to be come an issue. Are there any stats on the current disk usage being displayed?

- name: Set up ccache
uses: hendrikmuhs/ccache-action@v1.2
with:
key: ${{ matrix.os }}-${{ matrix.name }}-ccache
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation of the hash key is here:
https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching

The has key is matched from the beginning to the end, so it should start with the less specific an go to the more specific. I suggest the following:

ccache-v1-${{ matrix.os }}-${{ matrix.name }}-${{branch-name}}

Additionally with the raw action/cache there is a restore key field that can take multiple lines. This should have the above first then ccache-v1-${{ matrix.os }}-main so that is falls back to the cache on the main. I don't recall the more advance logic to get the "release" cache to work correctly for PR to the release branch.... maybe a variable for the target branch name would be appropriate as a heigher priority.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type:Infrastructure Infrastructure/ecosystem related changes, such as CMake or buildbots type:Performance Improvement in terms of compilation or execution time type:Testing Ensure that the purpose of a class is met/the results on a wide set of test cases are correct

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants