Automate version management with setuptools-scm and simplify release workflow

.github/workflows/publish.yml

@@ -0,0 +1,85 @@
+name: Publish to PyPI
+
+on:
+  release:
+    types: [published]
+  workflow_dispatch:  # Manual trigger for testing TestPyPI
+
+jobs:
+  build:
+    name: Build distribution
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read  # Required to checkout the repository
+
+    steps:
+    - uses: actions/checkout@v6
+      with:
+        fetch-depth: 0  # Required for setuptools-scm to get version from tags
+
+    - name: Set up Python
+      uses: actions/setup-python@v6
+      with:
+        python-version: "3.12"
+
+    - name: Install build dependencies
+      run: |
+        python -m pip install --upgrade pip
+        python -m pip install build
+
+    - name: Build package
+      run: python -m build
+
+    - name: Check package with twine (informational)
+      run: |
+        python -m pip install twine
+        python -m twine check dist/* || true
+
+    - name: Upload distribution artifacts
+      uses: actions/upload-artifact@v6
+      with:
+        name: python-package-distributions
+        path: dist/
+
+  publish-to-pypi:
+    name: Publish to PyPI
+    if: github.event_name == 'release'  # Skip on manual workflow_dispatch
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: pypi
+      url: https://pypi.org/p/pycortex
+    permissions:
+      id-token: write  # Required for trusted publishing
+
+    steps:
+    - name: Download distribution artifacts
+      uses: actions/download-artifact@v7
+      with:
+        name: python-package-distributions
+        path: dist/
+
+    - name: Publish to PyPI
+      uses: pypa/gh-action-pypi-publish@release/v1
+
+  publish-to-testpypi:
+    name: Publish to TestPyPI
+    needs: build
+    runs-on: ubuntu-latest
+    environment:
+      name: testpypi
+      url: https://test.pypi.org/p/pycortex
+    permissions:
+      id-token: write  # Required for trusted publishing
+
+    steps:
+    - name: Download distribution artifacts
+      uses: actions/download-artifact@v7
+      with:
+        name: python-package-distributions
+        path: dist/
+
+    - name: Publish to TestPyPI
+      uses: pypa/gh-action-pypi-publish@release/v1
+      with:
+        repository-url: https://test.pypi.org/legacy/

.gitignore

@@ -6,6 +6,9 @@ scripts
 cortex/formats.c
 cortex/openctm.c
 
+# Version file generated by setuptools-scm
+cortex/_version.py
+
 # Packages
 *.egg
 *.egg-info

RELEASE.md

@@ -5,61 +5,75 @@ This document describes how to release a new version of pycortex.
 ## Prerequisites
 
 - Push access to the main repository
-- PyPI publishing is handled automatically via GitHub Actions
+- Maintainer access to create GitHub releases
+- PyPI/TestPyPI environments must be configured with trusted publishing (done by repository admins)
 
-## Steps
+## Simplified Release Process
 
-### 1. Update the version number
+Version numbers are now automatically derived from git tags using `setuptools-scm`. You no longer need to manually edit version files.
 
-Edit `cortex/version.py` and change the version from development to release:
+### Steps
 
-```python
-# Change from:
-__version__ = '1.3.0.dev0'
+#### 1. Create a GitHub Release
 
-# To:
-__version__ = '1.2.12'
-```
+1. Go to https://github.com/gallantlab/pycortex/releases
+2. Click "Draft a new release"
+3. Click "Choose a tag" and type a new tag name following semantic versioning (e.g., `v1.3.0`)
+4. Click "Create new tag: vX.Y.Z on publish"
+5. Set the release title (e.g., "Version 1.3.0")
+6. Use "Generate release notes" to auto-populate from merged PRs, or write custom notes
+7. Click "Publish release"
 
-### 2. Commit the version change
+That's it! The GitHub Actions workflow will automatically:
+- Detect the new release event
+- Checkout the code with full git history
+- Use `setuptools-scm` to derive the version from the git tag
+- Build the source distribution and wheel
+- Publish to PyPI (from the `pypi` environment)
+- Publish to TestPyPI (from the `testpypi` environment)
 
-```bash
-git add cortex/version.py
-git commit -m "MNT version 1.2.12"
-```
+#### 2. Verify the Release
 
-### 3. Create and push the tag
+- Check that the GitHub Actions workflow completed successfully
+- Verify the package appears on PyPI: https://pypi.org/project/pycortex/
+- Verify the package appears on TestPyPI: https://test.pypi.org/project/pycortex/
+- Test installing the new version: `pip install --upgrade pycortex`
 
-Use an annotated tag:
+## Versioning
 
-```bash
-git tag -a 1.2.12 -m "Version 1.2.12"
-git push origin main
-git push origin 1.2.12
-```
+- Version numbers are automatically derived from git tags by `setuptools-scm`
+- Release versions: `vX.Y.Z` (e.g., `v1.3.0`) → published as `X.Y.Z` on PyPI
+- Development versions between releases: `X.Y.Z.devN` (automatically generated)
+- Use semantic versioning: MAJOR.MINOR.PATCH
 
-This triggers GitHub Actions to:
-- Build and publish the source distribution to PyPI
-- Build and deploy documentation to GitHub Pages
+## Manual Testing (TestPyPI only)
 
-### 4. Create GitHub Release (optional)
+To test the release process without publishing to PyPI:
 
-Go to https://github.com/gallantlab/pycortex/releases and create a new release from the tag. Use "Generate release notes" to auto-populate from merged PRs.
+1. Go to https://github.com/gallantlab/pycortex/actions/workflows/publish.yml
+2. Click "Run workflow"
+3. Select the branch to test
+4. Click "Run workflow"
 
-### 5. Bump back to development version
+This will build the package and publish only to TestPyPI, not to PyPI.
 
-```bash
-# Edit cortex/version.py back to dev version
-# e.g., __version__ = '1.3.0.dev0'
+## Troubleshooting
 
-git add cortex/version.py
-git commit -m "MNT back to dev [skip ci]"
-git push origin main
-```
+### Build fails due to shallow clone
 
-## Versioning
+The workflow uses `fetch-depth: 0` to ensure full git history is available for `setuptools-scm`. If builds fail with version detection errors, check that this setting is present in the workflow.
+
+### Version number is incorrect
+
+`setuptools-scm` derives versions from git tags. Ensure:
+- Tags follow the format `vX.Y.Z` or `X.Y.Z`
+- Tags are annotated tags (created with `git tag -a`)
+- The repository has at least one tag
+
+### Publishing fails
 
-- Release versions: `X.Y.Z` (e.g., `1.2.12`)
-- Development versions: `X.Y.Z.dev0` (e.g., `1.3.0.dev0`)
+Check that:
+- The PyPI/TestPyPI environments are configured in the repository settings
+- Trusted publishing is configured for this repository on PyPI/TestPyPI
+- The workflow has the necessary permissions (`id-token: write`)
 
-The version in `cortex/version.py` is the single source of truth, used by `setup.py` and documentation.

cortex/version.py

@@ -1,67 +1,14 @@
 """Defines version to be imported in the module and obtained from setup.py
 """
-# This file has been copied and modified from the DataLad codebase
-# https://github.com/datalad/datalad/blob/master/datalad/version.py
-# www.datalad.org
-# Many thanks to the DataLad developers
+# Version is now automatically managed by setuptools-scm
+# which generates cortex/_version.py during build
 
-import sys
-from os.path import lexists, dirname, join as opj, curdir
+try:
+    from cortex._version import version as __version__
+    __full_version__ = __version__
+except ImportError:
+    # Fallback for development environments without setuptools-scm
+    __version__ = '1.3.0.dev0'
+    __full_version__ = __version__
 
-# Hard coded version, to be done by release process,
-# it is also "parsed" (not imported) by setup.py, that is why assigned as
-# __hardcoded_version__ later and not vice versa
-#
-# NOTE this should have the format of
-# NEW_RELEASE.dev0
-# since the __version__ will be later automatically parsed by adding the git
-# commit hash
-# so valid versions will be 1.2.1.dev0, 1.4.dev0, etc
-__version__ = '1.3.0dev0'
 __hardcoded_version__ = __version__
-__full_version__ = __version__
-
-# NOTE: might cause problems with "python setup.py develop" deployments
-#  so I have even changed buildbot to use  pip install -e .
-moddir = dirname(__file__)
-projdir = curdir if moddir == 'cortex' else dirname(moddir)
-if lexists(opj(projdir, '.git')):
-    # If under git -- attempt to deduce a better "dynamic" version following git
-    try:
-        from subprocess import Popen, PIPE
-        # Note: Popen does not support `with` way correctly in 2.7
-        #
-        git = Popen(
-            ['git', 'describe', '--abbrev=4', '--dirty', '--match', r'[0-9]*\.*'],
-            stdout=PIPE, stderr=PIPE,
-            cwd=projdir
-        )
-        if git.wait() != 0:
-            raise OSError("Could not run git describe")
-        line = git.stdout.readlines()[0]
-        _ = git.stderr.readlines()
-        # Just take describe and replace initial '-' with .dev to be more "pythonish"
-        # Encoding simply because distutils' LooseVersion compares only StringType
-        # and thus misses in __cmp__ necessary wrapping for unicode strings
-        #
-        # remove the version from describe and stick the current version
-        # hardcoded in __version__
-        dev_suffix = line.strip().decode('ascii').split('-')
-        # in case we are at release, we shouldn't have any dev
-        if len(dev_suffix) > 1:
-            dev_suffix = dev_suffix[1:]
-            dev_suffix = ".dev{}".format('-'.join(dev_suffix))
-        else:
-            dev_suffix = ""
-        # remove dev suffix from hardcoded version
-        if 'dev' in __version__:
-            __version__ = '.'.join(__version__.split('.')[:-1])
-        # stick the automatically generated dev version
-        __full_version__ = __version__ + dev_suffix
-        # To follow PEP440 we can't have all the git fanciness
-        __version__ = __full_version__.split('-')[0]
-    except (SyntaxError, AttributeError, IndexError):
-        raise
-    except:
-        # just stick to the hard-coded
-        pass

pyproject.toml

@@ -1,9 +1,13 @@
 [build-system]
 # Minimum requirements for the build system to execute, according to PEP518
 # specification.
-requires = ["setuptools", "build", "numpy", "cython", "wheel"]
+requires = ["setuptools>=64", "setuptools-scm>=8", "build", "numpy", "cython", "wheel"]
 build-backend = "setuptools.build_meta"
 
+[tool.setuptools_scm]
+version_file = "cortex/_version.py"
+local_scheme = "no-local-version"
+
 [tool.codespell]
 skip = '.git,*.pdf,*.svg,*.css,*.min.*,*.gii,resources,OpenCTM-1.0.3,filestore,build,_build'
 check-hidden = true

setup.py

@@ -51,20 +51,7 @@ def run(self):
 ]
 
 
-# Modified from DataLad codebase to load version from pycortex/version.py
-def get_version():
-    """Load version from version.py without entailing any imports
-    Parameters
-    ----------
-    name: str
-      Name of the folder (package) where from to read version.py
-    """
-    # This might entail lots of imports which might not yet be available
-    # so let's do ad-hoc parsing of the version.py
-    with open(os.path.abspath('cortex/version.py')) as f:
-        version_lines = list(filter(lambda x: x.startswith('__version__'), f))
-    assert (len(version_lines) == 1)
-    return version_lines[0].split('=')[1].strip(" '\"\t\n")
+
 
 
 ctm = Extension('cortex.openctm', [
@@ -93,8 +80,7 @@ def get_version():
                     include_dirs=[numpy.get_include()])
 
 DISTNAME = 'pycortex'
-# VERSION needs to be modified under cortex/version.py
-VERSION = get_version()
+# VERSION is now automatically derived from git tags via setuptools-scm
 DESCRIPTION = 'Python Cortical mapping software for fMRI data'
 with open('README.md') as f:
     LONG_DESCRIPTION = f.read()
@@ -108,7 +94,6 @@ def get_version():
 
 
 setup(name=DISTNAME,
-      version=VERSION,
       description=DESCRIPTION,
       long_description=LONG_DESCRIPTION,
       long_description_content_type='text/markdown',