environment.yaml: mace-torch upgrades NumPy to 2.x and breaks the conda-installed OpenMM (ABI mismatch)

[luedsch]

Creating the environment from environment.yaml leaves a broken install. The pip: section (via mace-torch) upgrades NumPy to 2.x after conda has installed an OpenMM that was compiled against NumPy 1.x. The log shows this explicitly:

Attempting uninstall: numpy
  Found existing installation: numpy 1.26.4
  Uninstalling numpy-1.26.4:
    Successfully uninstalled numpy-1.26.4
...
Successfully installed ... numpy-2.2.6 ...

Any subsequent import openmm.app then fails with a NumPy 1.x/2.x ABI error:

A module that was compiled using NumPy 1.x cannot be run in NumPy 2.2.6 ...
ImportError: numpy.core.multiarray failed to import
  (in openmm/app/internal/xtc_utils/xtc.pyx while importing openmm.app)

I could workaround that error by pinning NumPy back down after the pip installs with pip install "numpy<2". Unfortunately, this caused another clash with matscipy, as shown here:

matscipy 1.2.0 requires numpy>=2.0.0, but you have numpy 1.26.4 which is incompatible

However, I am still able to pass the pytest provided in tests/test_alchemical_states.py with this workaround. Note that I had to install that package too, as this is not present in environment.yaml.

---

Minor / documentation note (not a bug)

On an HPC login node without a GPU driver, conda env create fails because the virtual package __cuda is absent, so the resolver refuses the CUDA PyTorch builds that sire requires. Setting CONDA_OVERRIDE_CUDA=12.6 lets the solve proceed. Maybe this saves users some time...

[JMorado]

Thanks for reporting this! As far as I know, this package has only been used by "family and friends", so I'm not surprised you've come across some installation issues and outdated examples. Sorry about that.

One of the main challenges has been putting together an environment that works reliably across different machines. For example, we ended up with some fairly complex installation recipes for HPC systems: https://github.com/michellab/fes-ml/tree/main/dev-tools/hpc. To be honest, the provided environment works on my machine, and I usually end up resorting to ad hoc installation recipes on other systems. The main source of friction has been sire, which has a number of dependency conflicts with some of the packages that fes-ml depends on.

I've been meaning to switch the installation from a Conda environment to Pixi, but I haven't had the time to work on it yet. That should also resolve the intertwined Conda/pip issues you've reported. The dependency on the MLPs should probably also be optional as the package can also be used to run plain MM free energy calculations.

Out of curiosity (and so I can help you a bit more), what type of free energy calculations are you looking to run?

Could you please try installing the environment using the YAML in #19 and let me know if this solves the issue?

[luedsch]

Could you please try installing the environment using the YAML in #19 and let me know if this solves the issue?

With the environment from #19 the conda issues on solving the environment are gone. Unfortunetaly, the pytest now fails, see pytest.log. There is no difference now whether I use CONDA_OVERRIDE_CUDA=12.6 or not. What version of python are you using? For me, Python 3.12.13 was chosen automatically. Pinning the exact version may help here?

Out of curiosity (and so I can help you a bit more), what type of free energy calculations are you looking to run?

We work a lot with the computation of solvation free energy with Gromacs using classical force fields, but want to step into the field of ML/MM (with electrostatic embedding), i.e. the full functionality of fes-ml is of interest here.

[JMorado]

Glad to know that the environment issues are mostly resolved. I've added pymbar to the YAML file for now, although I think it should become an optional dependency in the future, since some users may prefer to use their own scripts for computing free energies.

Regarding the failing tests, the issue appears to be related to a recent change in Sire (OpenBioSim/sire@b27c346), which now calculates the LJ LRC using a dedicated CustomVolumeForce. The tests pass if I disable the LRC, but I'm still working on a more robust solution.

The examples in #21 use OpenFFCreationStrategy (which I recommend, as I've tested it much more extensively) rather than SireCreationStrategy, so this issue with the failing tests does not affect them.