Merge pull request #322 from iiasa/issue/313

Adjust packaging of Git LFS artefacts

Author: khaeru

.github/workflows/publish.yaml

@@ -13,6 +13,7 @@ on:
 jobs:
   publish:
     uses: iiasa/actions/.github/workflows/publish.yaml@main
+    with:
+      lfs: true
     secrets:
       PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
-      TESTPYPI_TOKEN: ${{ secrets.TESTPYPI_TOKEN }}

doc/api/tools-costs.rst

@@ -105,6 +105,8 @@ The tool uses the following data sources for the regional differentiation of cos
 
 The tool also uses :mod:`.ssp.data` (via :func:`.exo_data.prepare_computer`) to adjust the costs of technologies based on GDP per capita.
 
+.. _costs-usage:
+
 Usage
 =====
 
@@ -121,7 +123,7 @@ Those settings include the following; click each for the full description, allow
    :attr:`~.Config.scenario_version`,
    :attr:`~.Config.base_year`,
    :attr:`~.Config.convergence_year`,
-   :attr:`~.Config.use_vintages`, 
+   :attr:`~.Config.use_vintages`,
    :attr:`~.Config.fom_rate`, and
    :attr:`~.Config.format`.
 
@@ -130,7 +132,10 @@ Those settings include the following; click each for the full description, allow
 - "inv_cost": the investment costs of the technologies in each region.
 - "fix_cost": the fixed O&M costs of the technologies in each region.
 
-To use the tool with the default settings, simply create a :class:`.Config` object and pass it as an argument to :func:`.create_cost_projections`::
+To use the tool, you **must** first obtain a copy of the SSP input data.
+See the documentation at :mod:`message_ix_models.project.ssp.data`.
+
+Next, create a :class:`.Config` object and pass it as an argument to :func:`.create_cost_projections`::
 
    from message_ix_models.tools.costs import Config, create_cost_projections
 
@@ -149,7 +154,7 @@ See the file :file:`message_ix_models/tools/costs/demo.py` for multiple examples
 
 
 .. note:: The data produced are for all valid combinations of :math:`(y^V, y^A)`—including those that are beyond the `technical_lifetime` of the |t| to which they apply.
-   This may produce large data frames, depending on the number of technologies, regions, and scenarios. 
+   This may produce large data frames, depending on the number of technologies, regions, and scenarios.
    At the moment, :mod:`.tools.costs` does not filter out these combinations.
    If this is problematic, the user may consider filtering the data for valid combinations of :math:`(y^V, y^A)`.
 

doc/data.rst

@@ -82,6 +82,8 @@ These are the preferred location for:
 
 - Outputs, such as data or plot files generated by reporting.
 - Data files not distributable with :mod:`message_ix_models`, for instance those with access conditions (registration, payment, etc.).
+
+  These include, among others, :mod:`.project.ssp.data`.
 - Caches: temporary data files used to speed up other code by avoiding repeat of slow operations.
 
 These kinds of data **must not** be committed to :mod:`message_ix_models`.
@@ -105,12 +107,26 @@ This location **should** be outside the Git-controlled directories for :mod:`mes
 In other words, users **should** at least use (2) or (3) to specify such directories.
 If not, they **may** use :file:`.gitignore` files to hide these from Git.
 
+.. _cache-data:
+
 (4B) Cache data
 ~~~~~~~~~~~~~~~
 
-Code **should** use :func:`.platformdirs.user_cache_path` to identify a system-specific path to a cache directory.
+User code **should** use :attr:`.Config.cache_path` (equivalently :py:`Context.core.cache_path`) to identify a system-specific path to a cache directory.
 For example:
 
+.. code-block:: python
+
+   from message_ix_models.util.config import Config
+
+   cfg = Config()
+
+   cfg.cache_path
+
+This is also a way to identify the cache path used on a particular system where :mod:`message_ix_models` is installed.
+
+Internal code that cannot access a :class:`~.util.config.Config` or :class:`.Context` instance **should** instead use :func:`platformdirs.user_cache_path` directly:
+
 .. code-block:: python
 
    from platformdirs import user_cache_path
@@ -125,7 +141,6 @@ For example:
    # Construct a file path within this directory
    p = dir_.joinpath("data-file-name.csv")
 
-
 General guidelines
 ==================
 

doc/project/ssp.rst

@@ -33,15 +33,20 @@ Data
 .. automodule:: message_ix_models.project.ssp.data
    :members:
 
-   Although free of charge, neither the 2017 or 2024 SSP data can be downloaded automatically.
-   Both sources require that users first submit personal information to register before being able to retrieve the data.
-   :mod:`message_ix_models` does not circumvent this requirement.
-   Thus:
+   Although free of charge, both the 2017 and 2024 SSP data are provided under licenses that prevent distribution with or automated retrieval by :mod:`message_ix_models`.
+   Both sources require that users first submit personal information to register before being able to retrieve the data,
+   and restrict the distribution of the complete, original files.
 
-   - A copy of the data are stored in :mod:`message_data`.
-   - :mod:`message_ix_models` contains only a ‘fuzzed’ version of the data (same structure, random values) for testing purposes.
+   Thus, :mod:`message_ix_models` contains only a ‘fuzzed’ version of the data (same structure, random values) for testing purposes.
 
-   .. todo:: Allow users without access to :mod:`message_data` to read a local copy of this data from a :attr:`.Config.local_data` subdirectory.
+   In order to use these data via the tools in this module, the user must obtain a copy of the files
+   —for instance, by manually completing the registration and download steps—
+   and place them in the directory :attr:`.Config.cache_path` (see :ref:`cache-data`)
+   or :file:`message_ix_models/data/ssp/` within the environment where :mod:`message_ix_models` is installed.
+   For :class:`SSPUpdate`, the specific file names required are the ones given by :data:`.pooch.SOURCE`::
+
+      1706548837040-ssp_basic_drivers_release_3.0_full.csv.gz
+      1710759470883-ssp_basic_drivers_release_3.0.1_full.csv.gz
 
    .. autosummary::
       SSPOriginal

doc/whatsnew.rst

@@ -103,6 +103,7 @@ Documentation
   :doc:`project/prisma`,
   :doc:`project/sparccle`, and
   :doc:`project/uptake` (:pull:`282`).
+- Expand the :ref:`costs-usage` section of the :mod:`.tools.costs` documentation to describe the requirement for SSP input data (:issue:`313`, :pull:`322`).
 
 v2025.1.10
 ==========

message_ix_models/project/ssp/data.py

@@ -18,24 +18,29 @@
 class SSPOriginal(ExoDataSource):
     """Provider of exogenous data from the original SSP database.
 
-    To use data from this source, call :func:`.exo_data.prepare_computer` with the
-    arguments:
+    This database is accessible from https://tntcat.iiasa.ac.at/SspDb/dsd.
 
-    - `source`: Any value from :data:`.SSP_2017` or equivalent string, for instance
-      "ICONICS:SSP(2017).2". The specific SSP for which data is returned is determined
-      from the value.
-    - `source_kw` including:
+    To use data from this source:
 
-      - "model": one of:
+    1. Read the general documentation for :mod:`.project.ssp.data`.
+    2. If necessary, obtain copy of the original data file(s).
+    3. Call :func:`.exo_data.prepare_computer` with the arguments:
 
-          - IIASA GDP
-          - IIASA-WiC POP
-          - NCAR
-          - OECD Env-Growth
-          - PIK GDP-32
+       - `source`: Any value from :data:`.SSP_2017` or equivalent string, for instance
+         "ICONICS:SSP(2017).2". The specific SSP for which data is returned is
+         determined from the value.
+       - `source_kw` including:
 
-      - "measure": The measures available differ according to the model; see the source
-        data for details.
+         - "model": one of:
+
+           - IIASA GDP
+           - IIASA-WiC POP
+           - NCAR
+           - OECD Env-Growth
+           - PIK GDP-32
+
+         - "measure": The measures available differ according to the model; see the
+           source data for details.
 
     Example
     -------
@@ -117,12 +122,17 @@ def __call__(self):
 class SSPUpdate(ExoDataSource):
     """Provider of exogenous data from the SSP Update database.
 
-    To use data from this source, call :func:`.exo_data.prepare_computer` with the
-    arguments:
+    This database is accessible from https://data.ece.iiasa.ac.at/ssp.
+
+    To use data from this source:
+
+    1. Read the general documentation for :mod:`.project.ssp.data`.
+    2. If necessary, obtain copy of the original data file(s).
+    3. Call :func:`.exo_data.prepare_computer` with the arguments:
 
-    - `source`: Any value from :data:`.SSP_2024` or equivalent string, for instance
-      "ICONICS:SSP(2024).2".
-    - `release`: One of "3.0.1", "3.0", or "preview".
+       - `source`: Any value from :data:`.SSP_2024` or equivalent string, for instance
+         "ICONICS:SSP(2024).2".
+       - `release`: One of "3.0.1", "3.0", or "preview".
 
     Example
     -------

message_ix_models/util/config.py

@@ -165,9 +165,12 @@ def hexdigest(self, length: int = -1) -> str:
 class Config:
     """Core/top-level settings for :mod:`message_ix_models` and :mod:`message_data`."""
 
-    #: Base path for cached data, e.g. as given by the :program:`--cache-path` CLI
-    #: option. Default: the directory :file:`message-ix-models` within the directory
-    #: given by :func:`.platformdirs.user_cache_path`.
+    #: Base path for cached data. By default, the directory :file:`message-ix-models`
+    #: within the directory given by :func:`.platformdirs.user_cache_path`. This **may**
+    #: be something like :file:`$HOME/.cache/message-ix-models/`.
+    #: The :program:`--cache-path` CLI option modifies this value.
+    #:
+    #: See also :ref:`cache-data`.
     cache_path: Optional[Path] = None
 
     #: Paths of files containing debug outputs. See

pyproject.toml

@@ -15,7 +15,6 @@ classifiers = [
   "Development Status :: 5 - Production/Stable",
   "Intended Audience :: Developers",
   "Intended Audience :: Science/Research",
-  "License :: OSI Approved :: Apache Software License",
   "Natural Language :: English",
   "Operating System :: OS Independent",
   "Programming Language :: Python",
@@ -29,6 +28,7 @@ classifiers = [
   "Topic :: Scientific/Engineering",
   "Topic :: Scientific/Engineering :: Information Analysis",
 ]
+license = "Apache-2.0"
 requires-python = ">=3.9"
 dependencies = [
   "click",