Fix cache invalidation for module binaries with Wave#6914
Fix cache invalidation for module binaries with Wave#6914FloWuenne wants to merge 1 commit intonextflow-io:masterfrom
Conversation
Signed-off-by: FloWuenne <flowuenne@gmail.com>
✅ Deploy Preview for nextflow-docs-staging canceled.
|
|
From the issue referenced #6128, someone wrote a minimal test pipeline: jashapiro/nextflow-resume-test. We can test this fix using the following procedure: Run once:
where Change the module binary and rerun with resume: Expected result with the fix: Before the fix, both processes could be restored from cache even after changing modules/test/resources/usr/bin/hello_test.sh. The included groovy test in the PR should emulate the same behaviour. I have tested the fix using a local build of Nextflow with these changes and that worked. @pditommaso Do you prefer the fix implementation via a different mechanism? If so, can you elaborate how? |
|
If am i not wrong the test does not use wave https://github.com/jashapiro/nextflow-resume-test |
|
Ok, managed to replicate:
|
|
@pditommaso Thanks for replicating! Anything else missing to get this merged? |
|
Yes, this PR requires merging this first #6927 |
Summary
Fix task cache invalidation when using module binaries with Wave.
Task hashing previously considered project-level
bin/scripts but not module-level binaries underresources/usr/bin, which could cause stale cached tasks to be reused after a module binary changed.This change makes task hashing resolve referenced executables from the task bin paths, including module binary directories, so module binary updates invalidate the cache as expected.
Closes #6128
Testing
TaskHasherTest./gradlew :nextflow:test --tests 'nextflow.processor.TaskHasherTest'