Versioning with _version_git breaks if softioc is installed

[MJGaughran]

If a module installs softioc, and also uses VersionGit for versioning, the version number appears to be set to that of softioc when making a test release. This may also affect released modules.

Steps to reproduce (using Diamond tools):

dls-start-new-module.py -a python3 -n dls_example_module
cd dls_example_module/
pipenv install -d
pipenv install softioc
dls-py3 install-into-prefix
ls prefix/lib/python3.7/site-packages/

This will result in a module which shows the version to be 3.1, which is the latest release of softioc:

dls_example_module  dls_example_module-3.1.dist-info

What I have found so far:

• It looks like _version_git in the your new module doesn't get called when dls-py3 install-into-prefix is called.
• On a clean repository, pipenv run python -m setup bdist_wheel will get the correct version (last tag + commit hash etc). If you run dls-py3 install-into-prefix first, the wheel will get stuck with the softioc version
• When running _version_git after install, it appears that GIT_REFS and GIT_SHA1 are set to the values from softioc
• This exists on (at least) versions 3.0, 3.0b2 and 3.1

[thomascobb]

It seems to work for me with /dls_sw/prod/tools/RHEL7-x86_64/defaults/bin/pipenv (output below). The only suspicious code is this:

# Place the directory containing _version_git on the path
for path, _, filenames in os.walk(os.path.dirname(os.path.abspath(__file__))):
    if "_version_git.py" in filenames:
        sys.path.append(path)
        break:

which appears in setup.py of every VersionGit enabled module. If __file__ in setup.py points to a directory that also contains a softioc directory, then it might find that instead. Can you think of any way that could happen?

Here's the output of it working for me:

tmc43@pc0105:~/python $ dls-start-new-module.py -a python3 -n dls_example_module

Making clean directory structure for dls_example_module
Initialising repo in /dls/science/users/tmc43/common/python/dls_example_module
Repository created.
Staging files...
Committing files to repo... commit msg: "Initial commit"

A new Python 3 module has been created in the dls_example_module directory.
Edit setup.cfg to make sure it has the correct metadata.
Python code should typically be added to the dls_example_module package.

tmc43@pc0105:~/python $ cd dls_example_module/
tmc43@pc0105:~/python/dls_example_module $ pipenv install -d
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success! 
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success! 
Updated Pipfile.lock (c0f625)!
Installing dependencies from Pipfile.lock (c0f625)...
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:03
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
tmc43@pc0105:~/python/dls_example_module 1m18s $ pipenv install softioc
Installing softioc...
Adding softioc to Pipfile's [packages]...
✔ Installation Succeeded 
Pipfile.lock (c0f625) out of date, updating to (2a61dc)...
Locking [dev-packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success! 
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success! 
Updated Pipfile.lock (2a61dc)!
Installing dependencies from Pipfile.lock (2a61dc)...
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
tmc43@pc0105:~/python/dls_example_module 1m17s $ dls-py3 install-into-prefix
Installing into /dls/science/users/tmc43/common/python/dls_example_module/prefix. This can be slow.
Creating lightweight-venv from Pipfile.lock ...
Completed after 5.5s
Installing into prefix using lightweight-venv ...
Completed after 4.7s
Installation complete after 0m 10s.
tmc43@pc0105:~/python/dls_example_module 11s $ ls prefix/lib/python3.7/site-packages/
dls_example_module  dls_example_module-0.0+untagged.g59eeebf.dirty.dist-info
tmc43@pc0105:~/python/dls_example_module $ which pipenv
/dls_sw/prod/tools/RHEL7-x86_64/defaults/bin/pipenv

[MJGaughran]

It appears that this issue only affects some people. Andy Wilson, and one of the grads (so should be a fairly clean setup) got what I had. Will Rogers and another grad got the same as you

[AlexanderWells-diamond]

I seem to get the wrong version info too. I ran your example a half dozen times and had the same result each time. So whatever it is appears to be deterministic.

[thomascobb]

@MJGaughran @AlexanderWells-diamond please can you try applying 3618d94 to your dls_example_module and see if it fixes the issue? If so I'll create a new version of VersionGit that addresses this, and apply it to pythonSoftIOC

[AlexanderWells-diamond]

Apply the given code change to setup.py in my newly generated example module fixes the issue for me.

I assume this fix will also end up in the template module as well?

[thomascobb]

yes

[MJGaughran]

Yes, those changes fix the issue, and I have already applied them to my other module. I'll let you close this ticket.

[thomascobb]

I've come up with a slightly safer approach in DiamondLightSource/versiongit#9 which I will release as VersionGit 2.0 as it drops python2 support

I'll apply this change for pythonSoftIOC as we're keeping python2 support for now