`minimal-toolchains` looks at recipes and not at what is installed

[mboisson]

When using minimal-toolchains, EasyBuild looks for an EasyConfig, without validating that there is a corresponding installed module. For example, we have a recipe that depends on

    ('METIS', '5.1.0'),

EasyBuild finds that the git repository has a recipe

METIS-5.1.0-GCCcore-7.3.0.eb

This recipe is however not installed on our system. We have

METIS-5.1.0-GCC-7.3.0.eb

installed instead. This last recipe is compatible with the recipe that we are trying to install now (which uses gompi,2018.3.312 which has GCC 7.3.0 in it).

With --minimal-toolchains, it barfs on the following :

== Running parse hook for METIS-5.1.0-GCC-7.3.0.eb...
== Running parse hook for SCOTCH-6.0.6-gompi-2018.3.312.eb...
== processing EasyBuild easyconfig /cvmfs/soft.computecanada.ca/easybuild/easyconfigs/o/OpenFOAM/OpenFOAM-v1812-gompi-2018.3.312.eb
== building and installing avx2/MPI/gcc7.3/openmpi3.1/openfoam/v1812...
== fetching files [skipped]
== creating build dir, resetting environment...
== backup of existing module file stored at /cvmfs/soft.computecanada.ca/easybuild/modules/2017/avx2/MPI/gcc7.3/openmpi3.1/openfoam/v1812.bak_20190912155905_5432
== unpacking [skipped]
== patching [skipped]
== preparing...
== FAILED: Installation ended unsuccessfully (build directory: /cvmfs/soft.computecanada.ca/easybuild/software/2017/avx2/MPI/gcc7.3/openmpi3.1/openfoam/v1812): build failed (first 300 chars): Missing modules for dependencies (use --robot?): Core/metis/5.1.0
== Results of the build can be found in the log file(s) /tmp/eb-9upFqW/easybuild-OpenFOAM-v1812-20190912.155905.JgRvr.log
ERROR: Build of /cvmfs/soft.computecanada.ca/easybuild/easyconfigs/o/OpenFOAM/OpenFOAM-v1812-gompi-2018.3.312.eb failed (err: 'build failed (first 300 chars): Missing modules for dependencies (use --robot?): Core/metis/5.1.0')

@bartoldeman

[ocaisa]

I think --use-existing-modules would solve this for you.

[mboisson]

That option indeed solves the issue. What other side effect does --use-existing-modules have though ?

[ocaisa]

None, it just means if you have an existing installation higher up the hierarchy it will prefer it.

It was created for zlib, which you would normally have at GCCcore but you also might want at iccifort level since it is highly tuned there.

[mboisson]

Does it still respect minimal toolchains within the installed modules ? Say there is a version with MPI and one without (for whatever reason).

[ocaisa]

In a hierarchy that's the same difference as GCCcore/iccifort so no, it would choose the MPI version (as long as the target software is using the MPI toolchain or higher). If there is some strange corner case where that's not appropriate, you could always explicitly indicate the required toolchain in the easyconfig.

[mboisson]

Ok. We can use use-existing-modules as a workaround, but I would argue this should be the default behavior, not an exception.

[ocaisa]

v4.0 is the time to make your case ;)

[mboisson]

@boegel, thoughts ?

[mboisson]

Without --use-existing-modules the --minimal-toolchains feature is broken as soon as a recipe that has a lower toolchain appears in the easyconfig repository, even if that recipe is not installed on the current system.

[ocaisa]

Actually, that doesn't sound right. It doesn't/shouldn't fail, it will just resolve to the GCCcore and then install it (if you use the robot). It's not true that once another recipe appears it will fail, in fact this is expected for use-existing-modules to do anything useful.

I see in your error it's looking for Core/metis/5.1.0, so with the system toolchain. That seems odd, what does a dry run look like, can that resolve?

[mboisson]

Correction. It will fail if you don't use --robot (we never use --robot).

[mboisson]

I guess my point is that "minimal toolchain" should not depend on existing recipes (i.e. in the repo), but rather on installed recipes if a match is found. Otherwise, if a new recipe is created at a the level of a lower toolchain, it will fail existing recipes until that new recipe is installed.

[akesandgren]

Is that what you want locally? Because here we definitely want it to find the minimal-toolchain recipe and build it... (using --robot of course)

[ocaisa]

The problem with setting use-existing-modules on by default is that it makes the order of commands matter for the end result. If something depends on zlib, which has easyconfigs for GCCcore and iccifort, I will get a different result with the same command depending on whether or not the iccifort zlib is installed or not.

For that reason I think the default as is is correct, and that it really is an "expert mode" option.

[mboisson]

@akesandgren yes. Here, we don't want to build software that is not needed. If there is already a match installed, it should get used. It should not install a new recipe that is "more minimal" just because somebody created a recipe for it.

@ocaisa, the problem with not having use-existing-modules is that the result will vary over time, based on what new recipe is added to the git repository. Once somebody adds METIS-5.1.0-GCCcore-7.3.0.eb there is no going back to using the (in my opinion) correct version METIS-5.1.0-GCC-7.3.0.eb and METIS-5.1.0-iccifort-2018.3.eb
This is precisely what happened to me. A recipe that used to work now ended up not working because it suddenly found METIS-5.1.0-GCCcore-7.3.0.eb that did not exist in the past.