Remove work directories for processes where input has a specific value #7123
valterschutz
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
My nextflow processes often invoke python scripts, which might change as I develop them. Thus, I often end up wanting to rerun a process but only for a very limited set of inputs.
Unlike #5308 and #6683 , I don't want to change the caching behavior. I just want to remove specific work directories, so that nextflow understands that some processes need to rerun.
I first propose that
nextflow cleanis extended with an option to invalidate a specific process, e.g.nextflow clean -process <process>. I currently achieve the same behavior by temporarily disabling thecachefor that specific process, but I think this would be a more convenient solution.My second proposal is that the above command is additionally extended to support the syntax
nextflow clean -process <process> -input-name <input_name> -input-value <input_value>. For the time being, I mostly care about this working with integers and strings.For my personal workflow --- which is largely based on training reinforcement learning agents --- this is (IMO) the biggest issue of Nextflow, which I am otherwise very impressed by. I'm surprised that I haven't found anyone else suggesting this yet. Am I missing something obvious?
Beta Was this translation helpful? Give feedback.
All reactions