Skip to content

Commit 72424a9

Browse files
lukpuehlukpueh
authored
Merge pull request #1946 from lukpueh/auto-release
Add GH workflow to build and release on GH and PyPI
2 parents 31ca674 + b99d043 commit 72424a9

File tree

5 files changed

+163
-51
lines changed

5 files changed

+163
-51
lines changed

‎.github/workflows/cd.yml

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
name: CD
2+
concurrency: cd
3+
4+
# Trigger workflow on any completed CI (see further checks below)
5+
on:
6+
workflow_run:
7+
workflows: [CI]
8+
types: [completed]
9+
10+
jobs:
11+
build:
12+
name: Build
13+
runs-on: ubuntu-latest
14+
# Skip unless CI was successful and ran on release tag, a ref starting with 'v'.
15+
# NOTE: We assume CI does not trigger on branches that start with 'v' (see #1961)
16+
if: >-
17+
github.event.workflow_run.conclusion == 'success' &&
18+
startsWith(github.event.workflow_run.head_branch, 'v')
19+
outputs:
20+
release_id: ${{ steps.gh-release.outputs.id }}
21+
steps:
22+
- name: Checkout release tag
23+
uses: actions/checkout@a12a3943b4bdde767164f792f33f40b04645d846
24+
with:
25+
ref: ${{ github.event.workflow_run.head_branch }}
26+
27+
- name: Set up Python
28+
uses: actions/setup-python@0ebf233433c08fb9061af664d501c3f3ff0e9e20
29+
with:
30+
python-version: '3.x'
31+
32+
- name: Install build dependency
33+
run: python3 -m pip install --upgrade pip build
34+
35+
- name: Build binary wheel and source tarball
36+
run: python3 -m build --sdist --wheel --outdir dist/ .
37+
38+
- id: gh-release
39+
name: Publish GitHub release candiate
40+
uses: softprops/action-gh-release@1e07f4398721186383de40550babbdf2b84acfc5
41+
with:
42+
name: ${{ github.event.workflow_run.head_branch }}-rc
43+
tag_name: ${{ github.event.workflow_run.head_branch }}
44+
body: "Release waiting for review..."
45+
files: dist/*
46+
47+
- name: Store build artifacts
48+
uses: actions/upload-artifact@6673cd052c4cd6fcf4b4e6e60ea986c889389535
49+
# NOTE: The GitHub release page contains the release artifacts too, but using
50+
# GitHub upload/download actions seems robuster: there is no need to compute
51+
# download URLs and tampering with artifacts between jobs is more limited.
52+
with:
53+
name: build-artifacts
54+
path: dist
55+
56+
release:
57+
name: Release
58+
runs-on: ubuntu-latest
59+
needs: build
60+
environment: release
61+
steps:
62+
- name: Fetch build artifacts
63+
uses: actions/download-artifact@fb598a63ae348fa914e94cd0ff38f362e927b741
64+
with:
65+
name: build-artifacts
66+
path: dist
67+
68+
- name: Publish binary wheel and source tarball on PyPI
69+
uses: pypa/gh-action-pypi-publish@717ba43cfbb0387f6ce311b169a825772f54d295
70+
with:
71+
user: __token__
72+
password: ${{ secrets.PYPI_API_TOKEN }}
73+
74+
- name: Finalize GitHub release
75+
uses: actions/github-script@9ac08808f993958e9de277fe43a64532a609130e
76+
with:
77+
script: |
78+
await github.rest.repos.updateRelease({
79+
owner: context.repo.owner,
80+
repo: context.repo.repo,
81+
release_id: '${{ needs.build.outputs.release_id }}',
82+
name: '${{ github.event.workflow_run.head_branch }}',
83+
body: 'See [CHANGELOG.md](https://github.com/' +
84+
context.repo.owner + '/' + context.repo.repo + '/blob/' +
85+
'${{ github.event.workflow_run.head_branch }}'+
86+
'/docs/CHANGELOG.md) for details.'
87+
})

‎.github/workflows/ci.yml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,13 @@
11
name: CI
22

33
on:
4+
# NOTE: CD relies on this configuration (see #1961)
45
push:
56
branches:
67
- develop
8+
tags:
9+
- v*
10+
711
pull_request:
812
workflow_dispatch:
913

‎docs/RELEASE.md

Lines changed: 46 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,39 +1,50 @@
11
# Release process
22

3-
* Ensure you have a backup of all working files and then remove files not tracked by git
4-
`git clean -xdf`. **NOTE**: this will delete all files in the tuf tree that aren't
5-
tracked by git
6-
* Ensure `docs/CHANGELOG.md` contains a one-line summary of each [notable
3+
4+
**Prerequisites (one-time setup)**
5+
6+
7+
1. Go to [PyPI management page](https://pypi.org/manage/account/#api-tokens) and create
8+
an [API token](https://pypi.org/help/#apitoken) with its scope limited to the tuf project.
9+
1. Go to [GitHub
10+
settings](https://github.com/theupdateframework/python-tuf/settings/environments),
11+
create an
12+
[environment](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#creating-an-environment)
13+
called `release` and configure [review
14+
protection](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#required-reviewers).
15+
1. In the environment create a
16+
[secret](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#environment-secrets)
17+
called `PYPI_API_TOKEN` and paste the token created above.
18+
19+
## Release
20+
21+
1. Ensure `docs/CHANGELOG.md` contains a one-line summary of each [notable
722
change](https://keepachangelog.com/) since the prior release
8-
* Update `tuf/__init__.py` to the new version number "A.B.C"
9-
* Test packaging, uploading to Test PyPI and installing from a virtual environment
10-
(ensure commands invoking `python` below are using Python 3)
11-
* Remove existing dist build dirs
12-
* Create source dist and wheel `python3 -m build`
13-
* Sign source dist `gpg --detach-sign -a dist/tuf-A.B.C.tar.gz`
14-
* Sign wheel `gpg --detach-sign -a dist/tuf-A.B.C-py3-none-any.whl`
15-
* Upload to test PyPI `twine upload --repository testpypi dist/*`
16-
* Verify the uploaded package at https://test.pypi.org/project/tuf/:
17-
Note that installing packages with pip using test.pypi.org is potentially
18-
dangerous (as dependencies may be squatted): download the file and install
19-
the local file instead.
20-
* Create a PR with updated `CHANGELOG.md` and version bumps
21-
* Once the PR is merged, pull the updated `develop` branch locally
22-
* Create a signed tag matching the updated version number on the merge commit
23+
2. Update `tuf/__init__.py` to the new version number `A.B.C`
24+
3. Create a PR with updated `CHANGELOG.md` and version bumps
25+
26+
➔ Review PR on GitHub
27+
28+
4. Once the PR is merged, pull the updated `develop` branch locally
29+
5. Create a signed tag for the version number on the merge commit
2330
`git tag --sign vA.B.C -m "vA.B.C"`
24-
* Push the tag to GitHub `git push origin vA.B.C`
25-
* Create a new release on GitHub, copying the `CHANGELOG.md` entries for the
26-
release
27-
* Create a package for the formal release
28-
(ensure commands invoking `python` below are using Python 3)
29-
* Remove existing dist build dirs
30-
* Create source dist and wheel `python3 -m build`
31-
* Sign source dist `gpg --detach-sign -a dist/tuf-A.B.C.tar.gz`
32-
* Sign wheel `gpg --detach-sign -a dist/tuf-A.B.C-py3-none-any.whl`
33-
* Upload to PyPI `twine upload dist/*`
34-
* Verify the package at https://pypi.org/project/tuf/ and by installing with pip
35-
* Attach both signed dists and their detached signatures to the release on GitHub
36-
* `verify_release` should be used to make sure the release artifacts match the
37-
git sources, preferably by another developer on a different machine.
38-
* Announce the release on [#tuf on CNCF Slack](https://cloud-native.slack.com/archives/C8NMD3QJ3)
39-
* Ensure [POUF 1](https://github.com/theupdateframework/taps/blob/master/POUFs/reference-POUF/pouf1.md), for the reference implementation, is up-to-date
31+
6. Push the tag to GitHub `git push origin vA.B.C`
32+
33+
*A push triggers the [CI workflow](.github/workfows/ci.yml), which, on success,
34+
triggers the [CD workflow](.github/workfows/cd.yml), which builds source dist and
35+
wheel, creates a preliminary GitHub release under `vA.B.C-rc`, and pauses for review.*
36+
37+
7. Run `verify_release --skip-pypi` locally to make sure a build on your machine matches
38+
the preliminary release artifacts published on GitHub.
39+
40+
➔ [Review *deployment*](https://docs.github.com/en/actions/managing-workflow-runs/reviewing-deployments)
41+
on GitHub
42+
43+
*An approval resumes the CD workflow to publish the release on PyPI, and to finalize the
44+
GitHub release (removes `-rc` suffix and updates release notes).*
45+
46+
8. `verify_release` may be used again to make sure the PyPI release artifacts match the
47+
local build as well.
48+
9. Announce the release on [#tuf on CNCF Slack](https://cloud-native.slack.com/archives/C8NMD3QJ3)
49+
10. Ensure [POUF 1](https://github.com/theupdateframework/taps/blob/master/POUFs/reference-POUF/pouf1.md),
50+
for the reference implementation, is up-to-date

‎tox.ini

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ commands =
3939

4040
[testenv:lint]
4141
changedir = {toxinidir}
42-
lint_dirs = tuf examples tests
42+
lint_dirs = tuf examples tests verify_release
4343
commands =
4444
black --check --diff {[testenv:lint]lint_dirs}
4545
isort --check --diff {[testenv:lint]lint_dirs}

‎verify_release

Lines changed: 25 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ Builds a release from current commit and verifies that the release artifacts
99
on GitHub and PyPI match the built release artifacts.
1010
"""
1111

12+
import argparse
1213
import json
1314
import os
1415
import subprocess
@@ -17,12 +18,12 @@ from filecmp import dircmp
1718
from tempfile import TemporaryDirectory
1819

1920
try:
21+
import build as _ # type: ignore
2022
import requests
21-
import build
2223
except ImportError:
23-
print ("Error: verify_release requires modules 'requests' and 'build':")
24-
print (" pip install requests build")
25-
exit(1)
24+
print("Error: verify_release requires modules 'requests' and 'build':")
25+
print(" pip install requests build")
26+
sys.exit(1)
2627

2728
# Project variables
2829
# Note that only these project artifacts are supported:
@@ -135,9 +136,17 @@ def progress(s: str) -> None:
135136

136137

137138
def main() -> int:
139+
parser = argparse.ArgumentParser()
140+
parser.add_argument(
141+
"--skip-pypi",
142+
action="store_true",
143+
dest="skip_pypi",
144+
help="Skip PyPI release check.",
145+
)
146+
args = parser.parse_args()
147+
138148
success = True
139149
with TemporaryDirectory() as build_dir:
140-
141150
progress("Building release")
142151
build_version = build(build_dir)
143152
finished(f"Built release {build_version}")
@@ -152,16 +161,17 @@ def main() -> int:
152161
if github_version != build_version:
153162
finished(f"WARNING: GitHub latest version is {github_version}")
154163

155-
progress("Checking PyPI latest version")
156-
pypi_version = get_pypi_pip_version()
157-
if pypi_version != build_version:
158-
finished(f"WARNING: PyPI latest version is {pypi_version}")
159-
160-
progress("Downloading release from PyPI")
161-
if not verify_pypi_release(build_version, build_dir):
162-
# This is expected while build is not reproducible
163-
finished("ERROR: PyPI artifacts do not match built release")
164-
success = False
164+
if not args.skip_pypi:
165+
progress("Checking PyPI latest version")
166+
pypi_version = get_pypi_pip_version()
167+
if pypi_version != build_version:
168+
finished(f"WARNING: PyPI latest version is {pypi_version}")
169+
170+
progress("Downloading release from PyPI")
171+
if not verify_pypi_release(build_version, build_dir):
172+
# This is expected while build is not reproducible
173+
finished("ERROR: PyPI artifacts do not match built release")
174+
success = False
165175

166176
progress("Downloading release from GitHub")
167177
if not verify_github_release(build_version, build_dir):

0 commit comments

Comments
 (0)