Skip to content

Run the backport bot in CI - #3416

Open
tianyiy-tim wants to merge 4 commits into
aws:backport-stack/publishfrom
tianyiy-tim:add-backport-ci
Open

Run the backport bot in CI#3416
tianyiy-tim wants to merge 4 commits into
aws:backport-stack/publishfrom
tianyiy-tim:add-backport-ci

Conversation

@tianyiy-tim

@tianyiy-tim tianyiy-tim commented Aug 10, 2026

Copy link
Copy Markdown
Member

Issues:

Addresses P425131803

Description of changes:

analyze, apply and publish still need someone to run them after a fix merges.

This pull request adds .github/workflows/backport-bot.yml, which runs all three automatically when a pull request labelled needs-backport merges. The decision to backport stays with the reviewers who apply the label, not with the tool.

It also adds the bot's own OIDC role to the CDK stack, chaining into the shared AwsLcGitHubActionsBedrockRole that #3376 renamed for exactly this.

Stacked on #3415. The base here is backport-stack/publish, a scaffolding branch holding the commits below it, so this diff is only the 7 files this change touches. I will retarget it to main as the stack lands. Please don't merge it into the scaffolding branch.

Call-outs:

  • Two jobs, and the split is the point. analyze is the only job that reaches the model and it has contents: read. publish is the only job that can write and it never reaches the model. So repository content, which the model reads, is never handled by a job holding a token that could change the repository. The verdict moves between them as an artifact.

  • The checkout is the merge commit, never the pull request head, so no untrusted code runs.

  • The role is pinned by job_workflow_ref to this one workflow file. The bot can't borrow autofix's role and general CI can't borrow either. Renaming the file breaks the trust policy until the stack is redeployed.

  • The pinning cuts both ways, following what autofix already does: the general AwsLcGitHubActionsOidcRole now excludes this workflow file too, so the bot can't skip its own role and assume the general one to reach the rest of CI. Synthesized the stack to confirm both exclusions land in the trust policy and that the Bedrock role ends up with one statement per OIDC role.

  • In CI the checkout already is aws/aws-lc, so the branches have nowhere else to go. --push-to-aws-lc says so out loud, is refused anywhere else, and push_branch only ever pushes a branch named backport-, so the escape has two independent limits.

  • Two things this needs that merging it doesn't provide, and I can't do either: the CDK stack has to be deployed before the workflow can assume the new role, and a pull request opened with GITHUB_TOKEN starts no workflow, so the backport pull requests would arrive with no CI of their own. Getting them tested needs a token that isn't GITHUB_TOKEN.

  • Uses Share Bedrock model settings between autofix and util/backport #3395 (merged) for .github/workflows/ai-config.json, which the workflow reads for the region.

Testing:

Unit tests - 6 new, 173 total:

python3 -m unittest testing.test_engine

They cover the --push-to-aws-lc escape, that the refusal still stands without it, and that only a backport- branch can be pushed.

CDK - synthesized the OIDC stack (aws-cdk-lib + constructs, no deploy) and read the rendered trust policies: AwsLcGitHubActionsBackportOidcRole pinned to backport-bot.yml@*, AwsLcGitHubActionsOidcRole excluding both autofix_integration_failures.yml@* and backport-bot.yml@*, and AwsLcGitHubActionsBedrockRole trusting the autofix and backport OIDC roles with sts:AssumeRole + sts:TagSession. Checked the staging account path renders aws-lc-staging in all of them.

Workflow - parsed it to confirm the two jobs hold the permissions above, analyze has no write access, publish has no id-token, the settings step runs before configure-aws-credentials, and the CDK pin matches this filename. I can't run the bot end to end until the role is deployed.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and the ISC license.

@tianyiy-tim
tianyiy-tim force-pushed the backport-stack/publish branch from f1eca9f to caf1347 Compare August 10, 2026 18:09
@tianyiy-tim
tianyiy-tim force-pushed the backport-stack/publish branch from caf1347 to e9becee Compare August 10, 2026 21:43
@tianyiy-tim
tianyiy-tim force-pushed the backport-stack/publish branch from e9becee to 7c1ee95 Compare August 10, 2026 22:20
@tianyiy-tim
tianyiy-tim force-pushed the backport-stack/publish branch from 7c1ee95 to c9dc173 Compare August 11, 2026 17:53
@tianyiy-tim
tianyiy-tim force-pushed the backport-stack/publish branch from c9dc173 to af8be4e Compare August 12, 2026 16:46
@tianyiy-tim
tianyiy-tim force-pushed the backport-stack/publish branch from af8be4e to db6b557 Compare August 13, 2026 18:26
@tianyiy-tim
tianyiy-tim force-pushed the add-backport-ci branch 3 times, most recently from a98ee5d to 8f25b64 Compare August 13, 2026 18:39
@tianyiy-tim
tianyiy-tim force-pushed the backport-stack/publish branch from db6b557 to dc7c482 Compare August 13, 2026 18:39
@tianyiy-tim
tianyiy-tim marked this pull request as ready for review August 13, 2026 19:19
@tianyiy-tim
tianyiy-tim requested a review from a team as a code owner August 13, 2026 19:19
@tianyiy-tim
tianyiy-tim force-pushed the backport-stack/publish branch from dc7c482 to 4396667 Compare August 13, 2026 19:23
Comment on lines +96 to +100
# The backport bot gets its own OIDC role, pinned to its own workflow file the
# same way autofix's role is above. Without a role of its own, any workflow in
# the repository could assume it and reach the shared Bedrock role below
backport_oidc_role_name = "AwsLcGitHubActionsBackportOidcRole"
self.backport_oidc_role = iam.Role(self, id=backport_oidc_role_name, role_name=backport_oidc_role_name,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to create a shared role with autofix to use bedrock for both?

A pull request labelled needs-backport now opens its own backport pull
requests when it merges, so the seven branch case needs nobody at a keyboard

Two jobs, and the split is the point. analyze is the only job that reaches the
model and it has contents: read. publish is the only job that can write and it
never reaches the model. Repository content, which the model reads, is
therefore never handled by a job holding a token that could change the
repository. The verdict moves between them as an artifact

The workflow assumes its own AwsLcGitHubActionsBackportOidcRole, pinned by
job_workflow_ref to this one file, and that role is what chains into the
shared AwsLcGitHubActionsBedrockRole that aws#3376 renamed for exactly this. The
bot cannot borrow autofix's role and general CI cannot borrow either

The pinning cuts both ways, which is the half of autofix's pattern that is
easy to miss. The general AwsLcGitHubActionsOidcRole now excludes this
workflow file too, so the bot cannot skip its own role and assume the general
one to reach the rest of CI

In CI the checkout already is aws/aws-lc, so the branches have nowhere else to
go. --push-to-aws-lc says so out loud, is refused anywhere else, and
push_branch will only ever push a branch named backport-, so the escape has
two independent limits rather than one

The checkout is the merge commit, never the pull request head, so no
untrusted code runs
@dougch
dougch self-requested a review August 13, 2026 23:47
Comment thread .github/workflows/backport-bot.yml Outdated
tianyiy-tim and others added 2 commits August 13, 2026 23:54
Co-authored-by: Doug Chapman <dougch@amazon.com>
The publish job was already moved to v7. v7 is what the rest of the repo uses,
157 workflows to 5, so the two jobs matching each other is the point
needs: analyze
permissions:
contents: write # to push the backport branches
pull-requests: write # to open and comment on the backport pull requests

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this to work, we need to turn on Allow GitHub Actions to create and approve pull requests under Settings → Actions → General -> Workflow Permissions -> Allow GitHub Actions to create and approve pull requests. is this on?

This poses a security risk due to a GHA having perms to open a PR. However, since this is for backports, which are already established fixes, it might not be a big risk, but for autofix, had a discussion with @justsmth and he recommended against not turning this on as it could open a can of worms. If this was discussed beforehand then ignore.

oidcRole: AwsLcGitHubActionsBackportOidcRole
roleName: AwsLcGitHubActionsBedrockRole
- name: Install the AI client
run: pip3 install --user anthropic boto3

@prasden prasden Aug 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pin to a major version release so any vulnerabilities can be tracked

Both run blocks interpolated ${{ }} straight into the shell, which is the
pattern GitHub's script injection guidance warns about: the expression is
expanded before the shell sees it, so the value becomes part of the command
rather than an argument to it. The two values here are a SHA and a number, so
there is nothing to inject today, but the shape is the finding

FIX_SHA and PR_NUMBER now arrive as environment variables and the scripts
quote them, matching what the rest of the repository does

Raised by AppSec against the backport-stack/ci branch, V2265233159
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants