From 3c285227dad5148b9381df1d649d235aadff4a12 Mon Sep 17 00:00:00 2001 From: Gil Forcada Codinachs Date: Sun, 26 Oct 2025 21:48:58 +0100 Subject: [PATCH 1/6] feat: explain how to switch to add native namespace To go along with the PLIP 3928. --- docs/developer-guide/index.md | 1 + docs/developer-guide/native-namespaces.md | 201 ++++++++++++++++++++++ 2 files changed, 202 insertions(+) create mode 100644 docs/developer-guide/native-namespaces.md diff --git a/docs/developer-guide/index.md b/docs/developer-guide/index.md index 0832fe252..34a35b79e 100644 --- a/docs/developer-guide/index.md +++ b/docs/developer-guide/index.md @@ -24,4 +24,5 @@ You can help consolidate all of development documentation here, even if it is to develop-volto-add-ons-index create-a-distribution standardize-python-project-configuration +native-namespaces ``` diff --git a/docs/developer-guide/native-namespaces.md b/docs/developer-guide/native-namespaces.md new file mode 100644 index 000000000..e3e33b0a5 --- /dev/null +++ b/docs/developer-guide/native-namespaces.md @@ -0,0 +1,201 @@ +# Native namespace + +This document explains the steps needed to convert a python distribution from `pkg_resources` namespace to native namespaces. + +## Background + +Python, since Python 3.3, added support for native namespaces, see [PEP 420](https://peps.python.org/pep-0420/). + +Plone has been using `pkg_resources`-style namespaces, but they are deprecated in `setuptools`. + +`setuptools` is planning to remove `pkg_resources`'s namespaces support by the end of 2025. + +[PLIP 3928](https://github.com/plone/Products.CMFPlone/issues/3928) tracks the changes needed for Plone to adapt to the _new_ native namespaces. + +## Steps + +To convert a given (`$package`) python distribution to use native namespaces follow these steps. + +### Create maintenance branch + +```{note} +Only relevant for Plone core packages +``` + +```{tip} +This part is only needed when the `main` or `master` branch is used on multiple versions of the Plone core development buildout. +``` + +Clone the repository or ensure you are at latest changes: + +```shell +git clone git@github.com:plone/$package +cd $package +# or update +git fetch -p +git checkout main # or master +git rebase +``` + +Find out what's the last release and create a branch for it: + +```shell +# list all tags +git for-each-ref --sort=taggerdate --format '%(tag)' refs/tags +# get the last tag's major number +MAJOR=`git for-each-ref --sort=taggerdate --format '%(tag)' refs/tags | tail -n1 | cut -d"." -f1` +# create a branch for it +git checkout -b $MAJOR.x +# push the newly created branch +git push +``` + +### Update buildout.coredev + +```{note} +Only relevant for Plone core packages +``` + +Update `buildout.coredev`'s branch 6.1 to use the newly created branch + +```shell +# go to the repository or clone it +cd buildout.coredev +# ensure you are at latest changes +git fetch -p +git checkout 6.1 +git rebase +# update the branch being used by the python distribution +sed -i "s/$package.git branch=master/$package.git branch=$MAJOR.x/" sources.cfg +# add the changes, commit and push +git add sources.cfg +git commit -m"chore: use branch $MAJOR.x for $package" +git push +``` + +Now check if you need to do the same on the 6.0 branch of `buildout.coredev`. + +```{tip} +You can use this [handy table](https://jenkins.plone.org/roboto/branches) to know which branch is used of a given package on each Plone version +``` + +```{tip} +To lower the amount of builds in Jenkins, either do a few at a time or add a `[ci-skip]` on the commit message +``` + +### Numbers before + +One risk of changing to the native namespaces is that some files, or tests, might be left behind. + +To ensure all tests and files are kept after the switch, gather the numbers before the change: + +Get the list of tests: + +```shell +tox run -e test -- --list-tests | wc -l +``` + +```{note} +Adapt to whichever way you are using to run the tests. +The above is meant for repositories that follow `plone.meta` conventions. +The `--list-tests` comes from `zope.testrunner`, if you are using that but not `plone.meta` you can use that as well. +``` + +Create a distribution to get a listing of how many files are currently packaged: + +```shell +rm -rf dist/ +uvx --from build pyproject-build +``` + +A `dist` folder is created with two archives in it: a `.tar.gz` and a `.whl`. + +To get the number of files on them run the following commands: + +```shell +python -c "import glob; import tarfile; print(len(tarfile.open(glob.glob('dist/*.tar.gz')[0], 'r:gz').getnames()))" +python -c "import glob; from zipfile import ZipFile; print(len(ZipFile(glob.glob('dist/*.whl')[0]).namelist()))" +``` + +Keep these numbers around for later. + +### Build backend + +To ensure the package continues to build, ensure that `setuptools` is defined as its build backend. + +For that inspect the `pyproject.toml`, it should have these lines: + +```toml +[build-system] +requires = ["setuptools>=68.2,<80", "wheel"] +``` + +If they are not there, add them, commit and push the changes. + +### Convert to native namespace + +Use `plone.meta`'s `switch-to-pep420` script: + +```shell +cd $package +uvx --from plone.meta switch-to-pep420 --no-tests . +``` + +```{tip} +This will also bump the version to a new major release. + +If the `main` or `master` branch is already an alpha version that is only used in Plone 6.2, you can specify that you don't want this version bump by adding the `--no-breaking` option. +``` + +### Update the test matrix + +```{note} +Only relevant for Plone core packages +``` + +As the switch to the native namespace has to be coordinated, all python distributions need to be only for the same Plone version, in this case it was decided to do it for the Plone 6.2 version. + +Thus, we need to ensure that the test matrix, only tests against this Plone version. + +For that, update `.meta.toml` with the following changes: + +```toml +[tox] +test_matrix = {"6.2" = ["*"]} +``` + +And update the scaffolding files with `plone.meta`: + +```shell +uvx --from plone.meta config-package --branch current . +``` + +Review the changes and ensure all changes are sound. + +```{note} +If the diff is quite big, run `config-package` before all the changes, get that on a Pull Request, approved and merged and then do a follow up Pull Request to move to native namespace. +``` + +### Compare numbers + +Get the list of tests, like before and compare the lists to ensure that the same amount of tests are found. + +```shell +tox run -e test -- --list-tests | wc -l +``` + +Likewise, create the distribution files and compare the numbers with the previous run: + +```shell +rm -rf dist/ +uvx --from build pyproject-build +python -c "import glob; import tarfile; print(len(tarfile.open(glob.glob('dist/*.tar.gz')[0], 'r:gz').getnames()))" +python -c "import glob; from zipfile import ZipFile; print(len(ZipFile(glob.glob('dist/*.whl')[0]).namelist()))" +``` + +It is okay if the numbers are slightly lower. +For obvious reasons the old source distribution will have one or more extra `__init__.py` files. +Both old distributions are expected to have an extra `namespace_packages.txt` file and possible an `x-nspkg.pth` file. +If you lose more than a few files though, something is wrong. + +If the numbers are close enough, review the changes once more, and push the branch with the changes for others to review it. From c274e9d67770ad2486274adc1cd464b7f0e9a5d3 Mon Sep 17 00:00:00 2001 From: Steve Piercy Date: Mon, 10 Nov 2025 11:19:58 -0800 Subject: [PATCH 2/6] Update docs/developer-guide/native-namespaces.md Co-authored-by: Maurits van Rees --- docs/developer-guide/native-namespaces.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/developer-guide/native-namespaces.md b/docs/developer-guide/native-namespaces.md index e3e33b0a5..d6407359f 100644 --- a/docs/developer-guide/native-namespaces.md +++ b/docs/developer-guide/native-namespaces.md @@ -155,7 +155,7 @@ Only relevant for Plone core packages As the switch to the native namespace has to be coordinated, all python distributions need to be only for the same Plone version, in this case it was decided to do it for the Plone 6.2 version. -Thus, we need to ensure that the test matrix, only tests against this Plone version. +Thus, we need to ensure that the test matrix only tests against this Plone version. For that, update `.meta.toml` with the following changes: From 72c9a1d3cc341a06929a66c811d64376f226d0f0 Mon Sep 17 00:00:00 2001 From: Steve Piercy Date: Mon, 10 Nov 2025 22:23:46 -0800 Subject: [PATCH 3/6] Add html_meta stuff --- .../{native-namespaces.md => native-namespace.md} | 9 +++++++++ 1 file changed, 9 insertions(+) rename docs/developer-guide/{native-namespaces.md => native-namespace.md} (94%) diff --git a/docs/developer-guide/native-namespaces.md b/docs/developer-guide/native-namespace.md similarity index 94% rename from docs/developer-guide/native-namespaces.md rename to docs/developer-guide/native-namespace.md index d6407359f..63f632292 100644 --- a/docs/developer-guide/native-namespaces.md +++ b/docs/developer-guide/native-namespace.md @@ -1,3 +1,12 @@ +--- +myst: + html_meta: + "description": "How to convert a Python distribution from a pkg_resources namespace to a native namespace" + "property=og:description": "How to convert a Python distribution from a pkg_resources namespace to a native namespace" + "property=og:title": "Native namespace" + "keywords": "Plone 6, developer guide, native namespaces, pkg_resources, Python" +--- + # Native namespace This document explains the steps needed to convert a python distribution from `pkg_resources` namespace to native namespaces. From c3ca0bdc0d6797d9315db5a89b7b5d11c88d72ae Mon Sep 17 00:00:00 2001 From: Steve Piercy Date: Mon, 10 Nov 2025 22:25:02 -0800 Subject: [PATCH 4/6] Condense sections and remove verbosity --- docs/developer-guide/native-namespace.md | 28 ++++++++++-------------- 1 file changed, 12 insertions(+), 16 deletions(-) diff --git a/docs/developer-guide/native-namespace.md b/docs/developer-guide/native-namespace.md index 63f632292..4c36e8ed8 100644 --- a/docs/developer-guide/native-namespace.md +++ b/docs/developer-guide/native-namespace.md @@ -9,23 +9,19 @@ myst: # Native namespace -This document explains the steps needed to convert a python distribution from `pkg_resources` namespace to native namespaces. +This chapter is guide for how to convert a Python distribution from a `pkg_resources` namespace to a native namespace. -## Background - -Python, since Python 3.3, added support for native namespaces, see [PEP 420](https://peps.python.org/pep-0420/). +Python 3.3 added support for native namespaces. +See [PEP 420](https://peps.python.org/pep-0420/) for more details. Plone has been using `pkg_resources`-style namespaces, but they are deprecated in `setuptools`. - -`setuptools` is planning to remove `pkg_resources`'s namespaces support by the end of 2025. - +`setuptools` is planning to remove `pkg_resources`'s namespace support by the end of 2025. [PLIP 3928](https://github.com/plone/Products.CMFPlone/issues/3928) tracks the changes needed for Plone to adapt to the _new_ native namespaces. -## Steps +To convert a given Python distribution to use a native namespace, follow these steps. -To convert a given (`$package`) python distribution to use native namespaces follow these steps. -### Create maintenance branch +## Create maintenance branch ```{note} Only relevant for Plone core packages @@ -59,7 +55,7 @@ git checkout -b $MAJOR.x git push ``` -### Update buildout.coredev +## Update `buildout.coredev` ```{note} Only relevant for Plone core packages @@ -92,7 +88,7 @@ You can use this [handy table](https://jenkins.plone.org/roboto/branches) to kno To lower the amount of builds in Jenkins, either do a few at a time or add a `[ci-skip]` on the commit message ``` -### Numbers before +## Numbers before One risk of changing to the native namespaces is that some files, or tests, might be left behind. @@ -128,7 +124,7 @@ python -c "import glob; from zipfile import ZipFile; print(len(ZipFile(glob.glob Keep these numbers around for later. -### Build backend +## Build backend To ensure the package continues to build, ensure that `setuptools` is defined as its build backend. @@ -141,7 +137,7 @@ requires = ["setuptools>=68.2,<80", "wheel"] If they are not there, add them, commit and push the changes. -### Convert to native namespace +## Convert to native namespace Use `plone.meta`'s `switch-to-pep420` script: @@ -156,7 +152,7 @@ This will also bump the version to a new major release. If the `main` or `master` branch is already an alpha version that is only used in Plone 6.2, you can specify that you don't want this version bump by adding the `--no-breaking` option. ``` -### Update the test matrix +## Update the test matrix ```{note} Only relevant for Plone core packages @@ -185,7 +181,7 @@ Review the changes and ensure all changes are sound. If the diff is quite big, run `config-package` before all the changes, get that on a Pull Request, approved and merged and then do a follow up Pull Request to move to native namespace. ``` -### Compare numbers +## Compare numbers Get the list of tests, like before and compare the lists to ensure that the same amount of tests are found. From 47d58a1a8954998b87ba1a504aeec540e21e7d60 Mon Sep 17 00:00:00 2001 From: Steve Piercy Date: Mon, 10 Nov 2025 22:27:02 -0800 Subject: [PATCH 5/6] Break sentence into two with relevant code blocks. Break another code block into separate code blocks for ease of copy-pasting into a shell session without comments. --- docs/developer-guide/native-namespace.md | 33 ++++++++++++++++++------ 1 file changed, 25 insertions(+), 8 deletions(-) diff --git a/docs/developer-guide/native-namespace.md b/docs/developer-guide/native-namespace.md index 4c36e8ed8..6e2af8560 100644 --- a/docs/developer-guide/native-namespace.md +++ b/docs/developer-guide/native-namespace.md @@ -24,34 +24,51 @@ To convert a given Python distribution to use a native namespace, follow these s ## Create maintenance branch ```{note} -Only relevant for Plone core packages +This step is relevant only for Plone core packages. ``` ```{tip} This part is only needed when the `main` or `master` branch is used on multiple versions of the Plone core development buildout. ``` -Clone the repository or ensure you are at latest changes: +If you haven't cloned the package's repository, then do so. ```shell git clone git@github.com:plone/$package cd $package -# or update +``` + +Otherwise, pull the latest changes into your local repository. + +```shell git fetch -p git checkout main # or master git rebase ``` -Find out what's the last release and create a branch for it: +Find the last release tag, and create a branch for it. + +List all tags. ```shell -# list all tags git for-each-ref --sort=taggerdate --format '%(tag)' refs/tags -# get the last tag's major number +``` + +Get the last tag's major number. + +```shell MAJOR=`git for-each-ref --sort=taggerdate --format '%(tag)' refs/tags | tail -n1 | cut -d"." -f1` -# create a branch for it +``` + +Create a branch for it. + +```shell git checkout -b $MAJOR.x -# push the newly created branch +``` + +Push the newly created branch. + +```shell git push ``` From 2798ee296f1fb0667373fd226b759cf3f6efce21 Mon Sep 17 00:00:00 2001 From: Steve Piercy Date: Mon, 10 Nov 2025 23:05:06 -0800 Subject: [PATCH 6/6] - Rename file to singular - Break up single code block into multiple to make comments more legible and code copy-pasteable. - Clean up grammar and enhance MyST syntax --- docs/developer-guide/index.md | 2 +- docs/developer-guide/native-namespace.md | 91 ++++++++++++++---------- 2 files changed, 54 insertions(+), 39 deletions(-) diff --git a/docs/developer-guide/index.md b/docs/developer-guide/index.md index 34a35b79e..940f95007 100644 --- a/docs/developer-guide/index.md +++ b/docs/developer-guide/index.md @@ -24,5 +24,5 @@ You can help consolidate all of development documentation here, even if it is to develop-volto-add-ons-index create-a-distribution standardize-python-project-configuration -native-namespaces +native-namespace ``` diff --git a/docs/developer-guide/native-namespace.md b/docs/developer-guide/native-namespace.md index 6e2af8560..c7bde3e96 100644 --- a/docs/developer-guide/native-namespace.md +++ b/docs/developer-guide/native-namespace.md @@ -72,46 +72,55 @@ Push the newly created branch. git push ``` + ## Update `buildout.coredev` ```{note} -Only relevant for Plone core packages +This step is relevant only for Plone core packages. ``` -Update `buildout.coredev`'s branch 6.1 to use the newly created branch +Update `buildout.coredev`'s branch `6.1` to use the newly created branch. ```shell -# go to the repository or clone it cd buildout.coredev -# ensure you are at latest changes +``` + +Ensure you have the latest changes. + +```shell git fetch -p git checkout 6.1 git rebase -# update the branch being used by the python distribution +``` + +Update the branch being used by the Python distribution. + +```shell sed -i "s/$package.git branch=master/$package.git branch=$MAJOR.x/" sources.cfg -# add the changes, commit and push +``` + +Add the changes, commit, and push. + +```shell git add sources.cfg git commit -m"chore: use branch $MAJOR.x for $package" git push ``` -Now check if you need to do the same on the 6.0 branch of `buildout.coredev`. +Now check if you need to do the same on the `6.0` branch of `buildout.coredev`. ```{tip} -You can use this [handy table](https://jenkins.plone.org/roboto/branches) to know which branch is used of a given package on each Plone version +You can use this [handy table](https://jenkins.plone.org/roboto/branches) to know which branch is used by a given package for each Plone version. ``` ```{tip} -To lower the amount of builds in Jenkins, either do a few at a time or add a `[ci-skip]` on the commit message +To lower the amount of builds in Jenkins, either do a few at a time or add a `[ci-skip]` on the commit message. ``` -## Numbers before - -One risk of changing to the native namespaces is that some files, or tests, might be left behind. +## Counts before change -To ensure all tests and files are kept after the switch, gather the numbers before the change: - -Get the list of tests: +There is a risk when changing to the native namespaces that some files, or tests, might be left behind. +To ensure all tests and files are kept after the switch, gather the counts before the change. ```shell tox run -e test -- --list-tests | wc -l @@ -120,43 +129,44 @@ tox run -e test -- --list-tests | wc -l ```{note} Adapt to whichever way you are using to run the tests. The above is meant for repositories that follow `plone.meta` conventions. -The `--list-tests` comes from `zope.testrunner`, if you are using that but not `plone.meta` you can use that as well. +The `--list-tests` comes from `zope.testrunner`, if you are using that, but not `plone.meta`, although you can use that as well. ``` -Create a distribution to get a listing of how many files are currently packaged: +Create a distribution to get a listing of how many files are currently packaged. ```shell rm -rf dist/ uvx --from build pyproject-build ``` -A `dist` folder is created with two archives in it: a `.tar.gz` and a `.whl`. - -To get the number of files on them run the following commands: +A `dist` folder is created with two archives in it, one each with the file suffix of `.tar.gz` and `.whl`. +To get the count of files in them, run the following commands. ```shell python -c "import glob; import tarfile; print(len(tarfile.open(glob.glob('dist/*.tar.gz')[0], 'r:gz').getnames()))" python -c "import glob; from zipfile import ZipFile; print(len(ZipFile(glob.glob('dist/*.whl')[0]).namelist()))" ``` -Keep these numbers around for later. +Note these counts for later in the step {ref}`compare-counts-after-change`. -## Build backend -To ensure the package continues to build, ensure that `setuptools` is defined as its build backend. +## Build backend -For that inspect the `pyproject.toml`, it should have these lines: +To ensure the package continues to build, verify that `setuptools` is defined as its build backend. +For that, inspect the {file}`pyproject.toml`. +It should have the following lines. ```toml [build-system] requires = ["setuptools>=68.2,<80", "wheel"] ``` -If they are not there, add them, commit and push the changes. +If they are not there, then add them, commit, and push the changes. + ## Convert to native namespace -Use `plone.meta`'s `switch-to-pep420` script: +Use `plone.meta`'s `switch-to-pep420` script. ```shell cd $package @@ -169,24 +179,24 @@ This will also bump the version to a new major release. If the `main` or `master` branch is already an alpha version that is only used in Plone 6.2, you can specify that you don't want this version bump by adding the `--no-breaking` option. ``` + ## Update the test matrix ```{note} -Only relevant for Plone core packages +This step is relevant only for Plone core packages. ``` -As the switch to the native namespace has to be coordinated, all python distributions need to be only for the same Plone version, in this case it was decided to do it for the Plone 6.2 version. - +Because the switch to the native namespace must be coordinated, all Python distributions need to be only for the same Plone version. +In this case, it was decided to do it for the Plone 6.2 version. Thus, we need to ensure that the test matrix only tests against this Plone version. - -For that, update `.meta.toml` with the following changes: +For that, update {file}`.meta.toml` with the following changes. ```toml [tox] test_matrix = {"6.2" = ["*"]} ``` -And update the scaffolding files with `plone.meta`: +Update the scaffolding files with `plone.meta`. ```shell uvx --from plone.meta config-package --branch current . @@ -195,18 +205,23 @@ uvx --from plone.meta config-package --branch current . Review the changes and ensure all changes are sound. ```{note} -If the diff is quite big, run `config-package` before all the changes, get that on a Pull Request, approved and merged and then do a follow up Pull Request to move to native namespace. +If the diff is quite big, then run `config-package` before all the changes. +Get the result into a pull request, approved, and merged. +Then do a follow up pull request to move it to a native namespace. ``` -## Compare numbers -Get the list of tests, like before and compare the lists to ensure that the same amount of tests are found. +(compare-counts-after-change)= + +## Compare counts after change + +Get the list of tests, and compare this result to the earlier one to ensure the same test counts. ```shell tox run -e test -- --list-tests | wc -l ``` -Likewise, create the distribution files and compare the numbers with the previous run: +Similarly, create the distribution files and compare the numbers with the previous run. ```shell rm -rf dist/ @@ -216,8 +231,8 @@ python -c "import glob; from zipfile import ZipFile; print(len(ZipFile(glob.glob ``` It is okay if the numbers are slightly lower. -For obvious reasons the old source distribution will have one or more extra `__init__.py` files. -Both old distributions are expected to have an extra `namespace_packages.txt` file and possible an `x-nspkg.pth` file. +For obvious reasons, the old source distribution will have one or more extra {file}`__init__.py` files. +Both old distributions are expected to have an extra {file}`namespace_packages.txt` file and possibly an {file}`x-nspkg.pth` file. If you lose more than a few files though, something is wrong. If the numbers are close enough, review the changes once more, and push the branch with the changes for others to review it.