Add gittools as varant to neptune-python-env/neptune-dev, add rank-run to base-env

[climbfuji]

Description

1. Add gittools variant (gh, py-pygithub) to neptune-python-env and enable in neptune-dev template. I couldn't enable it in the unified-dev template for oneapi, because the concretizer ended up in an infinite loop, heating my room overnight but not concretizing the environment.
2. Add rank-run to base-env.
3. Add external bash to all DOD HPCMP systems (required by gh --> go, don't want to build bash!), also find in GitHub workflows (I checked that it is being found and used).
4. Fix bug in .gitmodules: unintentional leftover from previous PR that didn't revert the spack submodule pointer information.

Dependencies

• Waiting on Add [email protected], new package rank-run spack-packages#36

Issues addressed

Closes #1805
Closes #1727

Applications affected

NEPTUNE

Systems affected

NRL systems

Testing

• CI: Note whether the automatic tests (GitHub actions tests that run automatically for every commit) pass or not
  • GitHub actions CI tests pass
  • GitHub actions CI tests do not pass (provide explanation)
  • GitHub actions CI tests skipped (provide explanation if necessary)
• New tests added: List and describe any new tests added to GitHub actions
  • ...
• Additional testing: Add information on any additional tests conducted
  • ...

Checklist

• This PR addresses one issue/problem/enhancement or has a very good reason for not doing so.
• These changes have been tested on the affected systems and applications.
• All dependency PRs/issues have been resolved and this PR can be merged.
• All necessary updates to the documentation (spack-stack wiki) will be made when this PR is merged

[AlexanderRichert-NOAA]

Which applications use rank-run? Is it needed for UFS Weather Model?

[climbfuji]

Which applications use rank-run? Is it needed for UFS Weather Model?

According to #1805, the rrfs-workflow uses it. It's such a trivial tool to install (dependencies, only one executable) that I don't have a problem including it in the base environment.

[AlexanderRichert-NOAA]

Separately from this PR I think it would pay to discuss base-env, what's in it, and whether we need it (i.e., as opposed to just separately enumerating dependencies for the apps that use it). My main concern is that at least currently, our main installations on WCOSS2 will be based on a trimmed down unified-dev template, but we're still constrained by what software is allowed on WCOSS2, so ideally I'd like the ufs-weather-model-env spec to correspond with just what UWM needs. I can create an issue and/or raise it in our next meeting.

[climbfuji]

Separately from this PR I think it would pay to discuss base-env, what's in it, and whether we need it (i.e., as opposed to just separately enumerating dependencies for the apps that use it). My main concern is that at least currently, our main installations on WCOSS2 will be based on a trimmed down unified-dev template, but we're still constrained by what software is allowed on WCOSS2, so ideally I'd like the ufs-weather-model-env spec to correspond with just what UWM needs. I can create an issue and/or raise it in our next meeting.

I thought about your question last night after I responded. That makes sense. We could create a variant with optional tools or define an env for rrfs-workflow and other workflows that don't have one yet, and then stick it in there. What do you think?

[AlexanderRichert-NOAA]

I'm okay with either solution, though I prefer the latter solution of separate metapackages as that feels more logical to me in terms of having metapackages that correspond with applications (though if we need a 'misc-env' catch-all metapackage I'm fine with that too). How does that sound?

[climbfuji]

I'm okay with either solution, though I prefer the latter solution of separate metapackages as that feels more logical to me in terms of having metapackages that correspond with applications (though if we need a 'misc-env' catch-all metapackage I'm fine with that too). How does that sound?

If I were to create misc-env, are there any other packages that should be moved from base-env to misc-env? I assume we then add misc-env to unified-dev, skylab-dev, and neptune-dev?

[AlexanderRichert-NOAA]

The packages that jump out at me are cloc, plus some of the Python stuff like pytest and setuptools. For the Python ones I could see either moving those to misc-env or adding a 'python-extras' variant. In my ideal world, spec'ing ufs-weather-model-env would provide strictly what's needed to build UFS Weather Model and run RTs (@rickgrubin @RatkoVasic-NOAA @natalie-perlin please chime in if you have another perspective).

[climbfuji]

The packages that jump out at me are cloc, plus some of the Python stuff like pytest and setuptools. For the Python ones I could see either moving those to misc-env or adding a 'python-extras' variant. In my ideal world, spec'ing ufs-weather-model-env would provide strictly what's needed to build UFS Weather Model and run RTs (@rickgrubin @RatkoVasic-NOAA @natalie-perlin please chime in if you have another perspective).

UFS is not unique here, same situation for NEPTUNE. We want a stripped-down environment to support exactly what is needed for NEPTUNE in operations - hence the neptune-ops template and all the variants and additional logic in neptune-env etc.

I wonder if a variant dev or devtools, shared between all our -env packages that need them, would be a possibility?

[AlexanderRichert-NOAA]

That sounds reasonable. I could live with setting '~devtools' or whatever in packages.yaml for Acorn/WCOSS2 for example.

[climbfuji]

That sounds reasonable. I could live with setting '~devtools' or whatever in packages.yaml for Acorn/WCOSS2 for example.

Ok. In this case, I suggest we move ahead with this PR as is (add rank-run to base-env) and discuss at the next spack-stack meeting the reorganization of the environments - which packages are part of the devtools variants for the different -envs that we share.

[AlexanderRichert-NOAA]

Sounds good 👍

[climbfuji]

Sounds good 👍

Thanks, could you also take a look at the submodule PR, please? JCSDA/spack-packages#36