|
| 1 | +<!-- |
| 2 | + Licensed to the Apache Software Foundation (ASF) under one |
| 3 | + or more contributor license agreements. See the NOTICE file |
| 4 | + distributed with this work for additional information |
| 5 | + regarding copyright ownership. The ASF licenses this file |
| 6 | + to you under the Apache License, Version 2.0 (the |
| 7 | + "License"); you may not use this file except in compliance |
| 8 | + with the License. You may obtain a copy of the License at |
| 9 | +
|
| 10 | + http://www.apache.org/licenses/LICENSE-2.0 |
| 11 | +
|
| 12 | + Unless required by applicable law or agreed to in writing, |
| 13 | + software distributed under the License is distributed on an |
| 14 | + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| 15 | + KIND, either express or implied. See the License for the |
| 16 | + specific language governing permissions and limitations |
| 17 | + under the License. |
| 18 | +--> |
| 19 | + |
| 20 | +# Release Verification Guide |
| 21 | + |
| 22 | +**Audience**: Committers and interested contributors. |
| 23 | + |
| 24 | +This guide walks you through the process of **verifying** a staged Apache Polaris release candidate. |
| 25 | + |
| 26 | +Verifying a (staged) release of an Apache project has to follow a bunch of tasks, which can be |
| 27 | +grouped into tasks that can be automated and those that need human intervention. |
| 28 | +Polaris provides a tool to automate the tasks that can be automated. |
| 29 | + |
| 30 | +Tasks that are automated: |
| 31 | +* Checksums and PGP signatures are valid. |
| 32 | +* All expected artifacts are present. |
| 33 | +* Source code artifacts have correct names matching the current release. |
| 34 | +* Built artifacts are [reproducible](#reproducible-builds). |
| 35 | +* Build passes. |
| 36 | +* `DISCLAIMER`, `LICENSE` and `NOTICE` files are included. |
| 37 | +* main and sources jar artifacts contain `META-INF/LICENSE` and `META-INF/NOTICE` files. |
| 38 | +* main distribution artifacts contain `DISCLAIMER`, `LICENSE` and `NOTICE` files in the top-level directory. |
| 39 | + |
| 40 | +Tasks that need human intervention: |
| 41 | +* Download links are valid. Check all links in the `[VOTE]` email for the release: |
| 42 | + * Tag on the GitHub website |
| 43 | + * Commit on the GitHub website |
| 44 | + * SVN repository with the source tarball and binary release artifacts |
| 45 | + * SVN repository with the Helm chart |
| 46 | + * Link to the KEYS file (_MUST_ be equal to `https://downloads.apache.org/incubator/polaris/KEYS`) |
| 47 | + * Maven staging repository |
| 48 | +* `DISCLAIMER`, `LICENSE` and `NOTICE` files are correct for the repository. |
| 49 | +* Contents of jar artifacts `META-INF/LICENSE` and `META-INF/NOTICE` files are correct. |
| 50 | +* All files have license headers if necessary. |
| 51 | + This is (mostly) verified using the "rat" tool during builds/CI. |
| 52 | +* No compiled artifacts are bundled in the source archive. |
| 53 | + This is a (soft) requirement to be held true by committers. |
| 54 | + |
| 55 | +**Imply good intent!** |
| 56 | +Although the release manager is responsible for producing a "proper" release, mistakes can and will happen. |
| 57 | +The Polaris project is committed to providing reproducible builds as an essential building block of |
| 58 | +_Apache trusted releases_. |
| 59 | +The project depends on frameworks which also strive to provide reproducible builds, but not all |
| 60 | +these frameworks can provide fully reproducible builds yet. |
| 61 | +The Polaris project's release verification tool will therefore report some issues that are currently expected. |
| 62 | +See [below](#reproducible-builds) for details. |
| 63 | + |
| 64 | +# Verifying a release candidate |
| 65 | + |
| 66 | +Instead of performing all mentioned steps manually, you can leverage the script |
| 67 | +`tools/verify-release/verify-release.sh` available in the main repository to perform the |
| 68 | +automatable tasks. |
| 69 | + |
| 70 | +Always run the most recent version of the script using the following command: |
| 71 | +```bash |
| 72 | +bash <(curl \ |
| 73 | + -s https://raw.githubusercontent.com/apache/polaris/refs/heads/main/tools/verify-release/verify-release.sh) \ |
| 74 | + --help |
| 75 | +``` |
| 76 | + |
| 77 | +The tool is intended for Polaris versions 1.3 and newer. |
| 78 | +The tool may report issues, see [below](#reproducible-builds) for details. |
| 79 | + |
| 80 | +That script requires a couple of tools installed and will check that those are available |
| 81 | +and report those that need to be installed. |
| 82 | + |
| 83 | +To run the script, you need the following pieces of information: |
| 84 | +* The *full* Git SHA of the corresponding source commit. |
| 85 | +* The version number of the release, something like `1.3.0` |
| 86 | +* The RC number of the release, for example `1` or `2` |
| 87 | +* The Maven staging repository ID, for example `1033` (the full number at the end of the Maven repository URL `https://repository.apache.org/content/repositories/orgapachepolaris-1033/`). |
| 88 | + |
| 89 | +Example (values taken from the 1.2.0-rc2 release) |
| 90 | +```bash |
| 91 | +bash <(curl \ |
| 92 | + -s https://raw.githubusercontent.com/apache/polaris/refs/heads/main/tools/verify-release/verify-release.sh) \ |
| 93 | + --git-sha 354a5ef6b337bf690b7a12fefe2c984e2139b029 \ |
| 94 | + --version 1.2.0 \ |
| 95 | + --rc 2 \ |
| 96 | + --maven-repo-id 1033 |
| 97 | +``` |
| 98 | + |
| 99 | +Same example, but using the short option names: |
| 100 | +```bash |
| 101 | +bash <(curl \ |
| 102 | + -s https://raw.githubusercontent.com/apache/polaris/refs/heads/main/tools/verify-release/verify-release.sh) \ |
| 103 | + -s 354a5ef6b337bf690b7a12fefe2c984e2139b029 \ |
| 104 | + -v 1.2.0 \ |
| 105 | + -r 2 \ |
| 106 | + -m 1033 |
| 107 | +``` |
| 108 | + |
| 109 | +The verification script creates a temporary directory that will eventually contain a fresh Git clone, |
| 110 | +all downloaded and all built artifacts. |
| 111 | +This temporary directory is deleted after the script has finished. To keep the temporary directory |
| 112 | +around, you can use the `--keep-temp-dir` (`-k`) option. |
| 113 | + |
| 114 | +A log file, the name matches the pattern `polaris-release-verify-*.log`, |
| 115 | +will be created and contain detailed information about the identified issues reported on the console. |
| 116 | + |
| 117 | +Note: The script is maintained in the Polaris source tree in the `/tools/verify-release` directory. |
| 118 | + |
| 119 | +## Verifications performed by the tool |
| 120 | + |
| 121 | +After some startup checks, the tool emits some information about the release candidate. For example: |
| 122 | +``` |
| 123 | +Verifying staged release |
| 124 | +======================== |
| 125 | +
|
| 126 | +Git tag: apache-polaris-1.2.0-incubating-rc2 |
| 127 | +Git sha: 354a5ef6b337bf690b7a12fefe2c984e2139b029 |
| 128 | +Full version: 1.2.0-incubating |
| 129 | +Maven repo URL: https://repository.apache.org/content/repositories/orgapachepolaris-1033/ |
| 130 | +Main dist URL: https://dist.apache.org/repos/dist/dev/incubator/polaris/1.2.0-incubating |
| 131 | +Helm chart URL: https://dist.apache.org/repos/dist/dev/incubator/polaris/helm-chart/1.2.0-incubating |
| 132 | +Verify directory: /tmp/polaris-release-verify-2025-10-23-14-22-31-HPmmiybzk |
| 133 | +
|
| 134 | +A verbose log containing the identified issues will be available here: |
| 135 | + /home/snazy/devel/polaris/polaris/polaris-release-verify-2025-10-23-14-22-31.log |
| 136 | +``` |
| 137 | + |
| 138 | +After that, release candidate verification starts immediately, performing the following operations: |
| 139 | + |
| 140 | +1. Create `gpg` keyring for signature verification from the project's `KEYS` file. |
| 141 | +2. Clone the Git repository directly from GitHub. |
| 142 | +3. Git commit SHA and Git tag match |
| 143 | +4. Verifies that the mandatory files are present in the source tree |
| 144 | +5. Helm chart GPG signature and checksum checks |
| 145 | +6. Source tarball and binary artifacts GPG signature and checksum checks |
| 146 | +7. Build Polaris from the Git commit/tag |
| 147 | +8. Compares that the list of Maven artifacts produced by the build matches those in the Nexus staging repository. |
| 148 | +9. Compares the individual Maven artifacts of the local build with the ones in the Nexus staging repository. |
| 149 | +10. Compares the main binary distribution artifacts for Polaris server and Polaris admin tool. |
| 150 | +11. Compares the locally built Helm chart with the one in the staging repository. |
| 151 | + |
| 152 | +Found issues are reported on the console in _red_. |
| 153 | + |
| 154 | +Details for each reported issue will be reported in the log file, depending on the issue and file type. |
| 155 | +The intent is to provide as much information as possible to eventually fix a reproducible build issue. |
| 156 | +* Text files: Output of `diff` of the local and staged files. |
| 157 | +* Zip/Jar files: output of `zipcmp`. |
| 158 | + If `zipcmp` reports no difference, the output of `zipinfo` for both files is logged. |
| 159 | +* Tarballs: output of `diff --recursive` the extracted local and staged tarballs. |
| 160 | + If `diff` reports no difference, the output of `tar tvf` for both files is logged. |
| 161 | +* Other files: Output of `diff` of the local and staged files. |
| 162 | + |
| 163 | +Note: GPG signatures are verified **only** against the project's `KEYS` file. |
| 164 | + |
| 165 | +# Reproducible builds |
| 166 | + |
| 167 | +A build is reproducible if the built artifacts are identical on every build from the same source. |
| 168 | + |
| 169 | +The Apache Polaris build is currently mostly reproducible, with some release-version specific exceptions. |
| 170 | + |
| 171 | +## Exceptions for all Apache Polaris versions |
| 172 | + |
| 173 | +Pending on full support for reproducible builds in Quarkus: |
| 174 | +* Jars containing generated code are not guaranteed to be reproducible. Affects the following jars: |
| 175 | + * */quarkus/generated-bytecode.jar |
| 176 | + * */quarkus/transformed-bytecode.jar |
| 177 | + * */quarkus/quarkus-application.jar |
| 178 | +* Re-assembled jars are not guaranteed to be reproducible: Affects the following jars: |
| 179 | + * admin/app/polaris-admin-*.jar |
| 180 | + * server/app/polaris-server-*.jar |
| 181 | +* Zips and tarballs containing any of the above are not guaranteed to be reproducible. |
| 182 | + |
| 183 | +Helm chart package tarball is not binary reproducible because there is no option to influence the |
| 184 | +mtime and POSIX attributes of the archive entries. |
| 185 | +The actual content of the archive entries is reproducible. |
| 186 | + |
| 187 | +## Exceptions for Apache Polaris up to 1.2 (including) |
| 188 | + |
| 189 | +* Depending on the operating system being used by the release manager and the "verifier," jar and zip files |
| 190 | + might be reported as different, even if the content of the jar and zip files is identical. |
| 191 | + This also leads to reported differences of the Gradle *.module files, because the checksums are different. |
| 192 | + Fixed via https://github.com/apache/polaris/pull/2819 |
| 193 | +* Source tarball is not binary reproducible because of non-constant mtime for tar entries. |
| 194 | + Fixed via https://github.com/apache/polaris/pull/2823 |
| 195 | +* The content of the parent pom contains dynamically generated content for the lists of developers and |
| 196 | + contributors. |
| 197 | + Fixed via https://github.com/apache/polaris/pull/2826 |
| 198 | + |
| 199 | +The following reported issues are expected, depending on the user's environment (`umask` likely). |
| 200 | +``` |
| 201 | +-------------------------------------------------------------------------- |
| 202 | + Comparing Maven repository artifacts, this will take a little while... |
| 203 | +-------------------------------------------------------------------------- |
| 204 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-admin/1.2.0-incubating/polaris-admin-1.2.0-incubating.module differ |
| 205 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-api-catalog-service/1.2.0-incubating/polaris-api-catalog-service-1.2.0-incubating.module differ |
| 206 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-api-iceberg-service/1.2.0-incubating/polaris-api-iceberg-service-1.2.0-incubating.module differ |
| 207 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-api-management-model/1.2.0-incubating/polaris-api-management-model-1.2.0-incubating.module differ |
| 208 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-api-management-service/1.2.0-incubating/polaris-api-management-service-1.2.0-incubating.module differ |
| 209 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-async-api/1.2.0-incubating/polaris-async-api-1.2.0-incubating.module differ |
| 210 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-async-java/1.2.0-incubating/polaris-async-java-1.2.0-incubating.module differ |
| 211 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-async-vertx/1.2.0-incubating/polaris-async-vertx-1.2.0-incubating.module differ |
| 212 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-config-docs-annotations/1.2.0-incubating/polaris-config-docs-annotations-1.2.0-incubating.module differ |
| 213 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-config-docs-generator/1.2.0-incubating/polaris-config-docs-generator-1.2.0-incubating.module differ |
| 214 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-container-spec-helper/1.2.0-incubating/polaris-container-spec-helper-1.2.0-incubating.module differ |
| 215 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-core/1.2.0-incubating/polaris-core-1.2.0-incubating.module differ |
| 216 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-eclipselink/1.2.0-incubating/polaris-eclipselink-1.2.0-incubating.module differ |
| 217 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-extensions-federation-hadoop/1.2.0-incubating/polaris-extensions-federation-hadoop-1.2.0-incubating.module differ |
| 218 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-extensions-federation-hive/1.2.0-incubating/polaris-extensions-federation-hive-1.2.0-incubating.module differ |
| 219 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-idgen-api/1.2.0-incubating/polaris-idgen-api-1.2.0-incubating.module differ |
| 220 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-idgen-impl/1.2.0-incubating/polaris-idgen-impl-1.2.0-incubating.module differ |
| 221 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-idgen-mocks/1.2.0-incubating/polaris-idgen-mocks-1.2.0-incubating.module differ |
| 222 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-idgen-spi/1.2.0-incubating/polaris-idgen-spi-1.2.0-incubating.module differ |
| 223 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-immutables/1.2.0-incubating/polaris-immutables-1.2.0-incubating.module differ |
| 224 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-minio-testcontainer/1.2.0-incubating/polaris-minio-testcontainer-1.2.0-incubating.module differ |
| 225 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-misc-types/1.2.0-incubating/polaris-misc-types-1.2.0-incubating.module differ |
| 226 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-persistence-nosql-varint/1.2.0-incubating/polaris-persistence-nosql-varint-1.2.0-incubating.module differ |
| 227 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-relational-jdbc/1.2.0-incubating/polaris-relational-jdbc-1.2.0-incubating.module differ |
| 228 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-runtime-common/1.2.0-incubating/polaris-runtime-common-1.2.0-incubating.module differ |
| 229 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-runtime-defaults/1.2.0-incubating/polaris-runtime-defaults-1.2.0-incubating.module differ |
| 230 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-runtime-service/1.2.0-incubating/polaris-runtime-service-1.2.0-incubating.module differ |
| 231 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-runtime-spark-tests/1.2.0-incubating/polaris-runtime-spark-tests-1.2.0-incubating.module differ |
| 232 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-runtime-test-common/1.2.0-incubating/polaris-runtime-test-common-1.2.0-incubating.module differ |
| 233 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-server/1.2.0-incubating/polaris-server-1.2.0-incubating.module differ |
| 234 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-spark-3.5_2.12/1.2.0-incubating/polaris-spark-3.5_2.12-1.2.0-incubating.module differ |
| 235 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-spark-3.5_2.13/1.2.0-incubating/polaris-spark-3.5_2.13-1.2.0-incubating.module differ |
| 236 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-spark-integration-3.5_2.12/1.2.0-incubating/polaris-spark-integration-3.5_2.12-1.2.0-incubating.module differ |
| 237 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-spark-integration-3.5_2.13/1.2.0-incubating/polaris-spark-integration-3.5_2.13-1.2.0-incubating.module differ |
| 238 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-tests/1.2.0-incubating/polaris-tests-1.2.0-incubating.module differ |
| 239 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris-version/1.2.0-incubating/polaris-version-1.2.0-incubating.module differ |
| 240 | +Locally built and staged Maven repository artifact org/apache/polaris/polaris/1.2.0-incubating/polaris-1.2.0-incubating.pom differ |
| 241 | +
|
| 242 | +
|
| 243 | +----------------------------------------- |
| 244 | + Comparing main distribution artifacts |
| 245 | +----------------------------------------- |
| 246 | +Locally built and staged Polaris distribution tarball polaris-bin-1.2.0-incubating.tgz differ |
| 247 | +Locally built and staged Polaris distribution zip polaris-bin-1.2.0-incubating.zip differ |
| 248 | +``` |
| 249 | + |
| 250 | +## Exceptions for Apache Polaris up to 1.1 (including) |
| 251 | + |
| 252 | +Apache Polaris builds up to 1.1 are not reproducible. |
0 commit comments