From 29bb590aa7addb27ffff528520bd53f4b624a4b4 Mon Sep 17 00:00:00 2001 From: Chad Bentz <1760475+felickz@users.noreply.github.com> Date: Mon, 6 Jul 2026 14:55:38 -0400 Subject: [PATCH 1/6] docs: clarify release/publish mechanics in CONTRIBUTING.md - Document how publish.yml decides what to publish (per-pack version diff vs GHCR) and that it only runs automatically on merge to main for packs whose version was bumped. - Explain that GitHub Releases / .release.yml / update-release.yml are a decoupled, manually-curated changelog and are NOT required for, or tied to, package publishing. - Document where to find what's published now (GHCR) vs what's about to publish (qlpack.yml on main), since no inventory exists. - Add a "Keeping CodeQL versions current" subsection describing today's manual CLI-update process and linking open PR #118. --- CONTRIBUTING.md | 80 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5e8667a1..655f84ca 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -85,6 +85,76 @@ Most of our query packs only depend on the standard library (`*-all`) for CodeQL > [!NOTE] > This table is maintained by hand today; update it whenever `.codeqlversion` or the `codeql-pack.lock.yml` files are refreshed. For a broader mapping of CodeQL CLI/bundle versions to per-language library versions (useful when triaging why a query compiles locally but not in CI, or vice versa), see the community [CodeQL Bundle Version Tracker](https://github.com/advanced-security/advanced-security-material/blob/main/codeql/codeql-version-tracker.md). +## Releases & publishing + +Publishing packages to the GitHub Container Registry (GHCR) and creating a GitHub Release are two +**separate, decoupled** processes in this repository. Understanding the difference matters if you're +bumping a pack's version or trying to find out what's actually live. + +### How packages get published + +[`publish.yml`][publish-workflow] runs on every push to `main` (i.e. every merge) plus manual +`workflow_dispatch`. For each language, and for each pack type it ships (`src` queries, `lib` libraries, +and — for `csharp`/`java` only today, see [#144][pr-144] — `ext` extensions and `ext-library-sources`), it: + +1. Reads the version currently published to GHCR for that pack. +2. Reads the local `version:` field from that pack's `qlpack.yml` on `main`. +3. Only runs `codeql pack install` + `codeql pack publish` if the two differ. + +This means publishing is **fully automatic on merge**, but **only for packs whose own `qlpack.yml` +`version:` field actually changed**. Merging a real change — a new query, a dependency/lock-file update, a +bug fix — does **not** publish anything on its own. You must also bump that pack's `version:` (following +[semver](https://semver.org/)), in the same PR or a fast-follow commit, or the change will sit on `main` +unpublished indefinitely. For example, removing a Java query required a separate follow-up commit bumping +`java/src/qlpack.yml`'s version before the change actually shipped to GHCR. + +> [!NOTE] +> A pack's dependents may also need a bump. For example `java/lib/qlpack.yml` pins +> `githubsecuritylab/codeql-java-extensions` to an exact version — if you bump `java/ext`, remember to +> update that pin too, or the two can drift out of sync (see [#155][pr-155]). + +There's no in-repo inventory of "what's currently published." To check: +- **What's live today** — the [GHCR Packages page][ghcr-packages] for this repo. +- **What will publish next** — the `version:` field in each language's `qlpack.yml` on `main`. + +### What GitHub Releases are for + +The [Releases][releases] tab (`v0.2.0`, `v0.2.1`, ...) is a **repo-wide changelog**, unrelated to the +per-pack publishing described above: + +- [`.release.yml`][release-config] is config for the [`42ByteLabs/patch-release-me`][patch-release-me] + tool. It tracks a single repository-wide `version:` and, when bumped, patches version references in + `configs/*.yml` (e.g. `owner/codeql-LANG-queries@version`) and dependency pins in `**/qlpack.yml` (e.g. + `owner/codeql-LANG-libs: version`) — it does **not** bump any pack's own top-level `version:` field. +- [`update-release.yml`][update-release-workflow] is a manual `workflow_dispatch` (pick + patch/minor/major) that runs that tool and opens a PR with the bumped `.release.yml` and patched + references. +- The GitHub Release itself is created manually afterwards by a maintainer via GitHub's "Draft a new + release" UI with auto-generated notes. + +> [!NOTE] +> These two systems are currently out of sync: `.release.yml`'s `version:` is still `0.2.0`, and the +> latest GitHub Release is `v0.2.1`, while individual packs (e.g. `codeql-java-queries`) are already +> published at `0.2.2`. Don't use the Releases tab or `.release.yml` to infer what's currently published — +> see "How packages get published" above for the real source of truth. + +### Keeping CodeQL versions current + +Bumping the CodeQL CLI/library version this repo builds against (tracked in [`.codeqlversion`][codeqlversion], +see [Supported CodeQL versions](#supported-codeql-versions) above) is a fully manual process today: + +1. Update `.codeqlversion` to the new CLI version. +2. Run `codeql pack upgrade ` for each pack directory to refresh its `codeql-pack.lock.yml`. +3. Fix any compilation/test errors caused by upstream API changes (usually the hardest part — see + [#124][pr-124] for an example). +4. Bump the `version:` field of every pack that changed, so `publish.yml` actually ships the update (see + "How packages get published" above). + +Nothing in the repo currently detects new upstream CodeQL CLI releases automatically — a maintainer has to +notice and kick off this process by hand. [#118][pr-118] proposes a weekly scheduled job to automate step +2 (`codeql pack upgrade`) only; steps 1, 3, and 4 would still need to be done manually even once it's +merged. + ## Using your personal data If you contribute to this project, we will record your name and email address (as provided by you with your contributions) as part of the code repositories, which are public. We might also use this information to contact you in relation to your contributions, as well as in the normal course of software development. We also store records of CLA agreements signed in the past, but no longer require contributors to sign a CLA. Under GDPR legislation, we do this on the basis of our legitimate interest in creating the CodeQL product. @@ -94,3 +164,13 @@ Please do get in touch (privacy@github.com) if you have any questions about this [codeqlversion]: ./.codeqlversion +[publish-workflow]: ./.github/workflows/publish.yml +[update-release-workflow]: ./.github/workflows/update-release.yml +[release-config]: ./.release.yml +[patch-release-me]: https://github.com/42ByteLabs/patch-release-me +[releases]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/releases +[ghcr-packages]: https://github.com/orgs/GitHubSecurityLab/packages?repo_name=CodeQL-Community-Packs +[pr-118]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/118 +[pr-124]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/124 +[pr-144]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/144 +[pr-155]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/155 From d259ac7b46df45f2f495993764fc98fb36207122 Mon Sep 17 00:00:00 2001 From: Chad Bentz <1760475+felickz@users.noreply.github.com> Date: Mon, 6 Jul 2026 14:58:57 -0400 Subject: [PATCH 2/6] ci: warn when pack content changes without a version bump Adds a new advisory (non-blocking) CI job, version-bump-check, that complements pr-suites-packs.sh's existing "version was bumped" comment: - pr-suites-packs.sh already comments when qlpack.yml/lock file changed AND the version differs from published (a bump was made). - This new check covers the opposite, previously-silent gap: files changed under /{src,lib,ext,ext-library-sources}/** but the pack's version still matches what's already published on GHCR, so publish.yml will not ship the change on merge. Deliberately advisory rather than blocking: the repo's actual practice sometimes splits a content change and its version bump across sequential PRs (see #123 -> #124), so a hard-blocking check would break that established workflow. --- .github/scripts/pr-version-bump-check.sh | 70 ++++++++++++++++++++++++ .github/workflows/ci.yml | 37 +++++++++++++ 2 files changed, 107 insertions(+) create mode 100644 .github/scripts/pr-version-bump-check.sh diff --git a/.github/scripts/pr-version-bump-check.sh b/.github/scripts/pr-version-bump-check.sh new file mode 100644 index 00000000..f58ac890 --- /dev/null +++ b/.github/scripts/pr-version-bump-check.sh @@ -0,0 +1,70 @@ +#!/bin/bash +set -euo pipefail + +# Warns (non-blocking) when a PR changes files inside a pack directory but +# does not bump that pack's own `version:` in qlpack.yml. +# +# publish.yml only republishes a pack when its qlpack.yml `version:` differs +# from what is currently on GHCR (see pr-suites-packs.sh for the companion +# check that fires when a bump *was* made). If content changes merge without +# a version bump, the change silently never ships - this script surfaces +# that gap as an advisory PR comment so it isn't forgotten. +# +# Usage: pr-version-bump-check.sh + +PR_NUMBER=${1} +LANGUAGE=${2} + +mapfile -t CHANGED_FILES < <(gh pr view "$PR_NUMBER" --json files --jq '.files.[].path') + +# subdir -> GHCR package name suffix (matches publish.yml) +declare -A PACKAGE_SUFFIX=( + [src]="queries" + [lib]="libs" + [ext]="extensions" + [ext-library-sources]="library-sources" +) + +for subdir in src lib ext ext-library-sources; do + qlpack_path="$LANGUAGE/$subdir/qlpack.yml" + if [[ ! -f "$qlpack_path" ]]; then + # Not every language has an ext / ext-library-sources pack. + continue + fi + + changed=false + for file in "${CHANGED_FILES[@]}"; do + if [[ "$file" == "$LANGUAGE/$subdir/"* ]]; then + changed=true + break + fi + done + + if [[ "$changed" != true ]]; then + continue + fi + + package="codeql-$LANGUAGE-${PACKAGE_SUFFIX[$subdir]}" + echo "[+] Files changed under $LANGUAGE/$subdir - checking whether $package's version was bumped" + + PUBLISHED_VERSION=$(gh api "/orgs/githubsecuritylab/packages/container/$package/versions" --jq '.[0].metadata.container.tags[0]' 2>/dev/null || echo "unknown") + CURRENT_VERSION=$(grep '^version:' "$qlpack_path" | awk '{print $2}' | tr -d '"'\''') + + if [[ "$PUBLISHED_VERSION" == "unknown" ]]; then + echo "[!] Could not resolve published version for $package - skipping (package may not exist yet)" + continue + fi + + if [[ "$PUBLISHED_VERSION" == "$CURRENT_VERSION" ]]; then + comment="Files changed in \`$LANGUAGE/$subdir\` but \`$qlpack_path\` version is still \`$CURRENT_VERSION\`, matching what is already published for \`$package\`. This change will NOT be published when this PR merges. If it should ship, bump \`version:\` in \`$qlpack_path\` (in this PR or a fast-follow)." + if [[ ! $(gh pr view "$PR_NUMBER" --json comments --jq '.comments.[].body' | grep "$comment") ]]; then + echo "[+] Commenting on PR: version bump appears to be missing for $package" + gh pr comment "$PR_NUMBER" \ + --body "$comment" + fi + else + echo "[+] $package version already bumped ($PUBLISHED_VERSION -> $CURRENT_VERSION), nothing to flag" + fi +done + +echo "[+] Complete" diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 53009e5b..c7aca6f2 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -251,3 +251,40 @@ jobs: GITHUB_TOKEN: ${{ github.token }} run: | ./.github/scripts/pr-configs.sh "${{ github.event.number }}" + + version-bump-check: + name: Check pack version was bumped + runs-on: ubuntu-latest + permissions: + contents: read + pull-requests: write # required by pr-version-bump-check.sh (gh pr comment) + issues: write # required by pr-version-bump-check.sh (gh pr comment) + packages: read # required by pr-version-bump-check.sh (gh api packages) + + strategy: + fail-fast: false + matrix: + language: [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ] + + steps: + - uses: actions/checkout@v7 + + - uses: dorny/paths-filter@7b450fff21473bca461d4b92ce414b9d0420d706 + id: changes + with: + filters: | + src: + - '${{ matrix.language }}/**' + + # Advisory only (never fails the build): publish.yml only republishes a + # pack when its qlpack.yml `version:` differs from what's on GHCR, so a + # content change that merges without a version bump silently never + # ships. This flags that gap on the PR instead of failing it, since the + # repo's actual practice sometimes intentionally splits a content + # change and its version bump across sequential PRs. + - name: "Check for forgotten version bump" + if: steps.changes.outputs.src == 'true' + env: + GITHUB_TOKEN: ${{ github.token }} + run: | + ./.github/scripts/pr-version-bump-check.sh "${{ github.event.number }}" "${{ matrix.language }}" From e20ef27add578fc03dd4271c154295e6061fb02d Mon Sep 17 00:00:00 2001 From: Chad Bentz <1760475+felickz@users.noreply.github.com> Date: Mon, 6 Jul 2026 15:01:18 -0400 Subject: [PATCH 3/6] ci: detect new upstream CodeQL CLI releases weekly Adds detect-codeql-release.yml, a weekly scheduled workflow (+ workflow_dispatch) that compares github/codeql-cli-binaries' latest release against .codeqlversion and, on drift, opens a tracking issue (idempotent - skipped if one is already open for that version) with the manual update checklist from CONTRIBUTING.md's "Keeping CodeQL versions current" section. Nothing in the repo previously watched for new upstream CodeQL releases at all - a maintainer had to notice by hand. This only automates detection + filing the issue; bumping .codeqlversion, running `codeql pack upgrade`, fixing any resulting compile/test breakage, and bumping affected pack versions remain manual follow-up steps for a maintainer or contributor to pick up (deliberately not automated further in this change). --- .github/scripts/check-codeql-release.sh | 64 +++++++++++++++++++++ .github/workflows/detect-codeql-release.yml | 22 +++++++ 2 files changed, 86 insertions(+) create mode 100644 .github/scripts/check-codeql-release.sh create mode 100644 .github/workflows/detect-codeql-release.yml diff --git a/.github/scripts/check-codeql-release.sh b/.github/scripts/check-codeql-release.sh new file mode 100644 index 00000000..fbb5d5a8 --- /dev/null +++ b/.github/scripts/check-codeql-release.sh @@ -0,0 +1,64 @@ +#!/bin/bash +set -euo pipefail + +# Checks whether a newer CodeQL CLI has been released upstream than the one +# pinned in .codeqlversion, and if so, opens a tracking issue with the manual +# update checklist from CONTRIBUTING.md ("Keeping CodeQL versions current"). +# +# Nothing else in the repo watches for new upstream CodeQL CLI releases today +# - this script/workflow only detects and files the issue. Actually bumping +# .codeqlversion, running `codeql pack upgrade`, fixing any resulting +# compile/test breakage, and bumping affected pack versions all remain +# manual steps for a maintainer/contributor to pick up. + +CURRENT_VERSION=$(cat .codeqlversion) +LATEST_TAG=$(gh api repos/github/codeql-cli-binaries/releases/latest --jq '.tag_name') +LATEST_VERSION=${LATEST_TAG#v} + +echo "[+] Currently pinned CodeQL CLI version: $CURRENT_VERSION" +echo "[+] Latest CodeQL CLI release upstream: $LATEST_VERSION" + +if [[ "$LATEST_VERSION" == "$CURRENT_VERSION" ]]; then + echo "[+] Already up to date, nothing to do." + exit 0 +fi + +echo "[+] New CodeQL CLI release detected: $CURRENT_VERSION -> $LATEST_VERSION" + +TITLE="CodeQL CLI v$LATEST_VERSION is available (currently pinned: v$CURRENT_VERSION)" + +# Idempotency: don't open a duplicate issue if one is already open for this version. +EXISTING=$(gh issue list --state open --search "\"$LATEST_VERSION\" in:title" --json number --jq 'length') +if [[ "$EXISTING" -gt 0 ]]; then + echo "[+] An open issue already mentions v$LATEST_VERSION in its title, skipping." + exit 0 +fi + +# Create the tracking label if it doesn't already exist (never fails the run). +if ! gh label list --search "codeql-cli-update" --json name --jq '.[].name' | grep -qx "codeql-cli-update"; then + gh label create "codeql-cli-update" --color "0E8A16" \ + --description "Tracks updating the pinned upstream CodeQL CLI/library version" || true +fi + +BINARY_URL="https://github.com/github/codeql-cli-binaries/releases/tag/$LATEST_TAG" +BUNDLE_URL="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v$LATEST_VERSION" + +BODY=$(cat <\` for each pack directory to refresh its \`codeql-pack.lock.yml\`. +- [ ] Fix any compilation/test errors caused by upstream API changes (usually the hardest part - see [#124](https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/124) for an example of what this can involve). +- [ ] Bump the \`version:\` field of every pack that changed, so \`publish.yml\` actually publishes the update once merged. +- [ ] Update the "Supported CodeQL versions" table in CONTRIBUTING.md. + +This issue was opened automatically by [\`detect-codeql-release.yml\`](../blob/main/.github/workflows/detect-codeql-release.yml). If you're not picking this up right away, feel free to leave it open as a tracker - a duplicate won't be filed while this stays open. +EOF +) + +echo "[+] Opening tracking issue" +gh issue create --title "$TITLE" --body "$BODY" --label "codeql-cli-update" diff --git a/.github/workflows/detect-codeql-release.yml b/.github/workflows/detect-codeql-release.yml new file mode 100644 index 00000000..cd3c3a93 --- /dev/null +++ b/.github/workflows/detect-codeql-release.yml @@ -0,0 +1,22 @@ +name: Detect new CodeQL CLI release + +on: + schedule: + - cron: '0 13 * * 1' # Every Monday at 13:00 UTC + workflow_dispatch: + +permissions: + contents: read + issues: write + +jobs: + check: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + + - name: Check for a new CodeQL CLI release + env: + GITHUB_TOKEN: ${{ github.token }} + run: | + ./.github/scripts/check-codeql-release.sh From ae6ed7fafdbdb15474ecc3763b4186ad505c9a06 Mon Sep 17 00:00:00 2001 From: Chad Bentz <1760475+felickz@users.noreply.github.com> Date: Mon, 6 Jul 2026 15:03:35 -0400 Subject: [PATCH 4/6] ci: generate a published-version inventory (PACKAGE_VERSIONS.md) Adds generate-version-inventory.sh (queries GHCR for every pack's published version, alongside the local qlpack.yml version) and a weekly scheduled workflow, publish-version-inventory.yml, that regenerates PACKAGE_VERSIONS.md and opens/updates a single PR when it drifts (reuses one stable branch, force-pushed, rather than piling up a new PR every week since the file is fully machine-generated). Today there's no in-repo inventory of what's published - answering "what's live?" means paging through the GHCR Packages UI or grepping 20+ qlpack.yml files by hand. PACKAGE_VERSIONS.md itself isn't included in this commit: it will be created by the first run of this workflow after merge (scheduled, or triggered on demand via workflow_dispatch). --- .github/scripts/generate-version-inventory.sh | 52 +++++++++++++++++ .../workflows/publish-version-inventory.yml | 57 +++++++++++++++++++ 2 files changed, 109 insertions(+) create mode 100644 .github/scripts/generate-version-inventory.sh create mode 100644 .github/workflows/publish-version-inventory.yml diff --git a/.github/scripts/generate-version-inventory.sh b/.github/scripts/generate-version-inventory.sh new file mode 100644 index 00000000..3d5974e1 --- /dev/null +++ b/.github/scripts/generate-version-inventory.sh @@ -0,0 +1,52 @@ +#!/bin/bash +set -euo pipefail + +# Generates a Markdown table of every pack's published (GHCR) version next +# to its local qlpack.yml version, so there's a single place to answer +# "what's live?" without paging through the GHCR Packages UI or grepping +# every qlpack.yml by hand (see CONTRIBUTING.md § How packages get +# published). +# +# Usage: generate-version-inventory.sh > PACKAGE_VERSIONS.md + +LANGUAGES=(cpp csharp go java javascript python ruby) + +# subdir -> (GHCR package suffix, human label) +declare -A PACKAGE_SUFFIX=( + [src]="queries" + [lib]="libs" + [ext]="extensions" + [ext-library-sources]="library-sources" +) +declare -A SUBDIR_LABEL=( + [src]="Queries (src)" + [lib]="Library (lib)" + [ext]="Extensions (ext)" + [ext-library-sources]="Library sources (ext-library-sources)" +) + +echo "" +echo "# Published package versions" +echo +echo "This table is regenerated on a schedule by [\`publish-version-inventory.yml\`](./.github/workflows/publish-version-inventory.yml)." +echo "\"Published\" is what's currently live on [GHCR](https://github.com/orgs/GitHubSecurityLab/packages?repo_name=CodeQL-Community-Packs)." +echo "\"On main\" is the version declared in that pack's \`qlpack.yml\` on the default branch - if it differs from" +echo "\"Published\", that version will be published the next time \`main\` is pushed to (see [publish.yml](./.github/workflows/publish.yml))." +echo +echo "| Language | Pack | Published | On main | Package |" +echo "| --- | --- | --- | --- | --- |" + +for language in "${LANGUAGES[@]}"; do + for subdir in src lib ext ext-library-sources; do + qlpack_path="$language/$subdir/qlpack.yml" + if [[ ! -f "$qlpack_path" ]]; then + continue + fi + + package="codeql-$language-${PACKAGE_SUFFIX[$subdir]}" + current_version=$(grep '^version:' "$qlpack_path" | awk '{print $2}' | tr -d '"'\''') + published_version=$(gh api "/orgs/githubsecuritylab/packages/container/$package/versions" --jq '.[0].metadata.container.tags[0]' 2>/dev/null || echo "_none_") + + echo "| $language | ${SUBDIR_LABEL[$subdir]} | \`$published_version\` | \`$current_version\` | [\`$package\`](https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pkgs/container/$package) |" + done +done diff --git a/.github/workflows/publish-version-inventory.yml b/.github/workflows/publish-version-inventory.yml new file mode 100644 index 00000000..4b8b2691 --- /dev/null +++ b/.github/workflows/publish-version-inventory.yml @@ -0,0 +1,57 @@ +name: Update published package version inventory + +on: + schedule: + - cron: '0 6 * * 1' # Every Monday at 06:00 UTC (offset from the other scheduled jobs) + workflow_dispatch: + +permissions: + contents: write + pull-requests: write + packages: read + +jobs: + update-inventory: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + + - name: Generate PACKAGE_VERSIONS.md + env: + GITHUB_TOKEN: ${{ github.token }} + run: | + ./.github/scripts/generate-version-inventory.sh > PACKAGE_VERSIONS.md + + - name: Check for changes + id: diff + run: | + if [[ -z "$(git status --porcelain -- PACKAGE_VERSIONS.md)" ]]; then + echo "changed=false" >> "$GITHUB_OUTPUT" + else + echo "changed=true" >> "$GITHUB_OUTPUT" + fi + + # Reuses one stable branch/PR (force-pushed each run) instead of opening + # a new PR every time the inventory drifts, since this file changes + # often (any time any pack is published) and is fully machine-generated. + - name: Open or update the inventory PR + if: steps.diff.outputs.changed == 'true' + env: + GITHUB_TOKEN: ${{ github.token }} + BRANCH: chore/package-inventory-update + run: | + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + + git checkout -b "$BRANCH" + git add PACKAGE_VERSIONS.md + git commit -m "chore: update published package version inventory" + git push --force origin "HEAD:$BRANCH" + + if [[ "$(gh pr list --head "$BRANCH" --state open --json number --jq 'length')" == "0" ]]; then + gh pr create --base main --head "$BRANCH" \ + --title "chore: update published package version inventory" \ + --body "Automated update of \`PACKAGE_VERSIONS.md\` (see [\`publish-version-inventory.yml\`](./.github/workflows/publish-version-inventory.yml))." + else + echo "[+] Existing PR for $BRANCH updated via force-push." + fi From 2f2710bee3ded7b27ac24fde9f8b86117d86ba48 Mon Sep 17 00:00:00 2001 From: Chad Bentz <1760475+felickz@users.noreply.github.com> Date: Mon, 6 Jul 2026 15:04:52 -0400 Subject: [PATCH 5/6] docs: document the new version-bump-check, release-detection, and inventory automation --- CONTRIBUTING.md | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 655f84ca..21e79de4 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -113,9 +113,17 @@ unpublished indefinitely. For example, removing a Java query required a separate > `githubsecuritylab/codeql-java-extensions` to an exact version — if you bump `java/ext`, remember to > update that pin too, or the two can drift out of sync (see [#155][pr-155]). -There's no in-repo inventory of "what's currently published." To check: -- **What's live today** — the [GHCR Packages page][ghcr-packages] for this repo. -- **What will publish next** — the `version:` field in each language's `qlpack.yml` on `main`. +Forgetting the version bump is easy, so `ci.yml` runs a **version-bump-check** job on every PR +([`pr-version-bump-check.sh`][version-bump-check-script]) that posts an advisory (non-blocking) comment if +files changed inside a pack directory but that pack's `version:` still matches what's already published. +It's advisory rather than blocking because content changes and their version bump are sometimes +intentionally split across sequential PRs (see [#123][pr-123] → [#124][pr-124]). + +There's an in-repo inventory of "what's currently published": [`PACKAGE_VERSIONS.md`][package-versions] is +regenerated weekly (and on demand) by [`publish-version-inventory.yml`][publish-version-inventory-workflow] +via [`generate-version-inventory.sh`][version-inventory-script]. To check: +- **What's live today** — [`PACKAGE_VERSIONS.md`][package-versions], or the [GHCR Packages page][ghcr-packages] for this repo directly. +- **What will publish next** — the `version:` field in each language's `qlpack.yml` on `main` (also shown in `PACKAGE_VERSIONS.md`'s "On main" column). ### What GitHub Releases are for @@ -155,6 +163,11 @@ notice and kick off this process by hand. [#118][pr-118] proposes a weekly sched 2 (`codeql pack upgrade`) only; steps 1, 3, and 4 would still need to be done manually even once it's merged. +To at least remove the "noticing" part, [`detect-codeql-release.yml`][detect-codeql-release-workflow] runs +weekly and opens a tracking issue (idempotently — it won't duplicate one that's already open) with this +checklist whenever `github/codeql-cli-binaries` has a newer release than [`.codeqlversion`][codeqlversion]. +It only detects and files the issue — steps 1–4 above are still manual work for whoever picks it up. + ## Using your personal data If you contribute to this project, we will record your name and email address (as provided by you with your contributions) as part of the code repositories, which are public. We might also use this information to contact you in relation to your contributions, as well as in the normal course of software development. We also store records of CLA agreements signed in the past, but no longer require contributors to sign a CLA. Under GDPR legislation, we do this on the basis of our legitimate interest in creating the CodeQL product. @@ -171,6 +184,12 @@ Please do get in touch (privacy@github.com) if you have any questions about this [releases]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/releases [ghcr-packages]: https://github.com/orgs/GitHubSecurityLab/packages?repo_name=CodeQL-Community-Packs [pr-118]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/118 +[pr-123]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/123 [pr-124]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/124 [pr-144]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/144 [pr-155]: https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/155 +[version-bump-check-script]: ./.github/scripts/pr-version-bump-check.sh +[package-versions]: ./PACKAGE_VERSIONS.md +[publish-version-inventory-workflow]: ./.github/workflows/publish-version-inventory.yml +[version-inventory-script]: ./.github/scripts/generate-version-inventory.sh +[detect-codeql-release-workflow]: ./.github/workflows/detect-codeql-release.yml From c2344ba4ea91c5757b4519ae941988c65e697dee Mon Sep 17 00:00:00 2001 From: Chad Bentz <1760475+felickz@users.noreply.github.com> Date: Mon, 6 Jul 2026 17:28:11 -0400 Subject: [PATCH 6/6] fix: address PR review feedback - pr-version-bump-check.sh: only check ext/ext-library-sources for csharp/java, matching publish.yml's actual matrix. go/ext and python/ext exist but aren't published by publish.yml (see #144), so telling contributors a version bump would ship them was misleading. - check-codeql-release.sh: replace broken `../blob/main/...` relative links in the generated issue body with full https://github.com/... URLs (derived from $GITHUB_REPOSITORY), since GitHub issue markdown resolves relative links against the issue's own path, not the repo root. - ci.yml: rename the advisory job from "Check pack version was bumped" to "Warn if pack version bump is missing" so the Actions UI doesn't imply the version was actually bumped when this job runs/passes. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> --- .github/scripts/check-codeql-release.sh | 8 +++++--- .github/scripts/pr-version-bump-check.sh | 13 ++++++++++++- .github/workflows/ci.yml | 2 +- 3 files changed, 18 insertions(+), 5 deletions(-) diff --git a/.github/scripts/check-codeql-release.sh b/.github/scripts/check-codeql-release.sh index fbb5d5a8..bcd289dc 100644 --- a/.github/scripts/check-codeql-release.sh +++ b/.github/scripts/check-codeql-release.sh @@ -42,13 +42,15 @@ fi BINARY_URL="https://github.com/github/codeql-cli-binaries/releases/tag/$LATEST_TAG" BUNDLE_URL="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v$LATEST_VERSION" +REPO="${GITHUB_REPOSITORY:-GitHubSecurityLab/CodeQL-Community-Packs}" +REPO_BLOB_URL="https://github.com/$REPO/blob/main" BODY=$(cat <\` for each pack directory to refresh its \`codeql-pack.lock.yml\`. @@ -56,7 +58,7 @@ Updating is a manual process today (see [CONTRIBUTING.md § Keeping CodeQL versi - [ ] Bump the \`version:\` field of every pack that changed, so \`publish.yml\` actually publishes the update once merged. - [ ] Update the "Supported CodeQL versions" table in CONTRIBUTING.md. -This issue was opened automatically by [\`detect-codeql-release.yml\`](../blob/main/.github/workflows/detect-codeql-release.yml). If you're not picking this up right away, feel free to leave it open as a tracker - a duplicate won't be filed while this stays open. +This issue was opened automatically by [\`detect-codeql-release.yml\`]($REPO_BLOB_URL/.github/workflows/detect-codeql-release.yml). If you're not picking this up right away, feel free to leave it open as a tracker - a duplicate won't be filed while this stays open. EOF ) diff --git a/.github/scripts/pr-version-bump-check.sh b/.github/scripts/pr-version-bump-check.sh index f58ac890..41236e34 100644 --- a/.github/scripts/pr-version-bump-check.sh +++ b/.github/scripts/pr-version-bump-check.sh @@ -25,7 +25,18 @@ declare -A PACKAGE_SUFFIX=( [ext-library-sources]="library-sources" ) -for subdir in src lib ext ext-library-sources; do +# publish.yml's `extensions`/`library_sources_extensions` jobs only matrix over +# csharp/java today (see #144 for the proposal to extend this to python/go). +# go/ext and python/ext directories exist but aren't wired into publish.yml, +# so bumping their version wouldn't actually make them publish - don't check +# those pack types for other languages, to avoid telling contributors a +# version bump will ship something that publish.yml doesn't yet publish. +SUBDIRS=(src lib) +if [[ "$LANGUAGE" == "csharp" || "$LANGUAGE" == "java" ]]; then + SUBDIRS+=(ext ext-library-sources) +fi + +for subdir in "${SUBDIRS[@]}"; do qlpack_path="$LANGUAGE/$subdir/qlpack.yml" if [[ ! -f "$qlpack_path" ]]; then # Not every language has an ext / ext-library-sources pack. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index c7aca6f2..63c32182 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -253,7 +253,7 @@ jobs: ./.github/scripts/pr-configs.sh "${{ github.event.number }}" version-bump-check: - name: Check pack version was bumped + name: Warn if pack version bump is missing runs-on: ubuntu-latest permissions: contents: read