Skip to content

Create pre-production stage for CI pipeline #2282

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Create pre-production stage for CI pipeline #2282

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
45a1b91
Create pipeline
nhatnghiho Feb 5, 2025
310e375
Update cdk instructions
nhatnghiho Mar 26, 2025
4fadbc4
Add typing and cleanup
nhatnghiho Mar 26, 2025
10d511f
Reformat code
nhatnghiho Mar 26, 2025
7ec6289
Fix minor bugs
nhatnghiho Mar 26, 2025
775eeed
Add step to auto-sync private repo
nhatnghiho Mar 28, 2025
3537e22
Fix dev pipeline deployment bug
nhatnghiho Apr 2, 2025
0a19c45
More cleanup
nhatnghiho Apr 2, 2025
1d84eea
Incorporate comments
nhatnghiho Apr 17, 2025
cf7cdeb
Remove stale benchmark scripts
nhatnghiho Apr 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 0 additions & 108 deletions tests/ci/build_run_benchmarks.sh
View file
Open in desktop

This file was deleted.

79 changes: 68 additions & 11 deletions tests/ci/cdk/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,26 @@

AWS-LC CI uses AWS CDK to define and deploy AWS resources (e.g. AWS CodeBuild, ECR).

## CI Setup
## Table of Contents
- [CDK Setup](#cdk-setup)
- [Before running CDK command](#before-running-cdk-command)
- [Minimal permissions](#minimal-permissions)
- [Pipeline Commands](#pipeline-commands)
- [CI Commands](#ci-commands)
- [AWS-LC Benchmarking Framework](#aws-lc-benchmarking-framework)
- [Framework Setup](#framework-setup)
- [How to Use](#how-to-use)
- [Start from Pull Request](#start-from-pull-request)
- [Start Locally](#start-locally)
- [Examine Output](#examine-output)
- [Files](#files)
- [Development Reference](#development-reference)
- [Useful commands](#useful-commands)
- [Useful Docker image build commands](#useful-docker-image-build-commands)
- [Linux Docker image build](#linux-docker-image-build)
- [Windows Docker image build (DEPRECATED)](#windows-docker-image-build-deprecated)

## CDK Setup

### Before running CDK command:

Expand Down Expand Up @@ -63,17 +82,50 @@ To setup or update the CI in your account you will need the following IAM permis
* secretsmanager:DeleteSecret
* secretsmanager:GetSecretValue

### Commands
### Pipeline Commands
Use these commands to deploy the CI pipeline. Any changes to the CI or Docker images will be updated automatically after the pipeline is deployed.

These commands are run from `aws-lc/tests/ci/cdk`.

[SKIP IF NO CROSS-ACCOUNT DEPLOYMENT] Give the pipeline account administrator access to the deployment account's CloudFormation. Repeat this step depending on how many deployment environment there are. You only need to run this step once when the pipeline is deploying to a new account for the first time.
```
cdk bootstrap aws://${DEPLOY_ACCOUNT_ID}/us-west-2 --trust ${PIPELINE_ACCOUNT_ID} --trust-for-lookup ${PIPELINE_ACCOUNT_ID} --cloudformation-execution-policies arn:aws:iam::aws:policy/AdministratorAccess
```

If not done previously, bootstrap cdk for the pipeline account before running the next commands.
```
cdk bootstrap aws://${PIPELINE_ACCOUNT_ID}/us-west-2
```

To deploy dev pipeline to the same account as your CI:
```
./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --github-source-version ${GITHUB_SOURCE_VERSION} --deploy-account ${DEPLOY_ACCOUNT_ID} --action deploy-dev-pipeline
```

To deploy dev pipeline but pipeline is hosted in a separate account:
```
./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --github-source-version ${GITHUB_SOURCE_VERSION} --pipeline-account ${PIPELINE_ACCOUNT_ID} --deploy-account ${DEPLOY_ACCOUNT_ID} --action deploy-dev-pipeline
```

To deploy production pipeline using default parameters:
```
./run-cdk.sh --action deploy-production-pipeline
```

### CI Commands
Use these commands if you wish to deploy individual stacks instead of the entire pipeline.

These commands are run from `aws-lc/tests/ci/cdk`.

If not done previously, bootstrap cdk before running the commands below. Make sure that AWS_ACCOUNT_ID is the AWS account you wish to deploy the CI stacks to.
Comment on lines +115 to +120
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any reason to keep this support and documented?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is helpful if we want to deploy 1 stack at a time. Say, if I only want to deploy aws-lc-ci-integration and don't want to deal with anything else, these commands will come in handy.


These commands are run from `aws-lc/tests/ci/cdk`. \
If not done previously, bootstrap cdk before running the commands below:
```shell
cdk bootstrap aws://${AWS_ACCOUNT_ID}/us-west-2
```

You may also need to request an increase to certain account quotas:
```shell
open https://${CDK_DEPLOY_REGION}.console.aws.amazon.com/servicequotas/home/services/ec2/quotas
open https://${DEPLOY_REGION}.console.aws.amazon.com/servicequotas/home/services/ec2/quotas
```
* **EC2-VPC Elastic IPs** = 20

Expand All @@ -82,23 +134,23 @@ Note: `GITHUB_REPO_OWNER` specifies the GitHub repo targeted by this CI setup.

To set up AWS-LC CI, run command:
```
./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --action deploy-ci --aws-account ${AWS_ACCOUNT_ID}
./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --action deploy-ci --deploy-account ${AWS_ACCOUNT_ID}
```

To update AWS-LC CI, run command:
```
./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --action update-ci --aws-account ${AWS_ACCOUNT_ID}
./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --action update-ci --deploy-account ${AWS_ACCOUNT_ID}
```

To create/update Linux Docker images, run command:
```
./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --action build-linux-img --aws-account ${AWS_ACCOUNT_ID}
./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --action build-linux-img --deploy-account ${AWS_ACCOUNT_ID}
```

To destroy AWS-LC CI resources created above, run command:
```
# NOTE: this command will destroy all resources (AWS CodeBuild and ECR).
./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --action destroy-ci --aws-account ${AWS_ACCOUNT_ID}
./run-cdk.sh --github-repo-owner ${GITHUB_REPO_OWNER} --action destroy-ci --deploy-account ${AWS_ACCOUNT_ID}
```

For help, run command:
Expand Down Expand Up @@ -155,6 +207,10 @@ Below is CI file structure.
│   ├── __init__.py
│   ├── ecr_stack.py
│   ├── ...
├── pipeline
│   ├── __init__.py
│   ├── pipeline_stack.py
│   ├── ...
├── cdk.json
├── requirements.txt
├── run-cdk.sh
Expand All @@ -167,7 +223,8 @@ Below is CI file structure.
* `README.md` — The introductory README for this project.
* `app.py` — The “main” for this sample application.
* `cdk.json` — A configuration file for CDK that defines what executable CDK should run to generate the CDK construct tree.
* `cdk` — A CDK module directory
* `cdk` — A module directory that contains all CI-related stacks and utilities
* `pipeline` - A module directory that defines a continuous deployment pipeline for the CI.
* `requirements.txt` — This file is used by pip to install all of the dependencies for your application. In this case, it contains only -e . This tells pip to install the requirements specified in setup.py. It also tells pip to run python setup.py develop to install the code in the cdk module so that it can be edited in place.
* `setup.py` — Defines how this Python package would be constructed and what the dependencies are.

Expand Down Expand Up @@ -244,7 +301,7 @@ aws codebuild start-build-batch --project-name aws-lc-docker-image-build-linux
# Go to AWS console, you can check CodeBuild by clicking "Developer Tools > CodeBuild > Build projects".
```

#### Windows Docker image build
#### Windows Docker image build (DEPRECATED)
Windows docker image build requires more resources (like EC2 host, S3, SSM and so on) set up because DIND (Docker in Docker) is not supported by Windows.
Below are some commands specific to windows docker image build.

Expand Down
79 changes: 35 additions & 44 deletions tests/ci/cdk/app.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,56 +5,47 @@

from aws_cdk import Environment, App

# from cdk.bm_framework_stack import BmFrameworkStack
from cdk.aws_lc_analytics_stack import AwsLcGitHubAnalyticsStack
from cdk.aws_lc_android_ci_stack import AwsLcAndroidCIStack
from cdk.aws_lc_github_ci_stack import AwsLcGitHubCIStack
from cdk.aws_lc_github_fuzz_ci_stack import AwsLcGitHubFuzzCIStack
from cdk.aws_lc_ec2_test_framework_ci_stack import AwsLcEC2TestingCIStack
from cdk.linux_docker_image_batch_build_stack import LinuxDockerImageBatchBuildStack
from pipeline.ci_util import add_ci_stacks
from pipeline.pipeline_stack import AwsLcCiPipeline
from cdk.windows_docker_image_build_stack import WindowsDockerImageBuildStack
from cdk.aws_lc_github_ci_x509_stack import AwsLcGitHubX509CIStack
from cdk.ecr_stack import EcrStack
from util.metadata import AWS_ACCOUNT, AWS_REGION, LINUX_X86_ECR_REPO, LINUX_AARCH_ECR_REPO, WINDOWS_X86_ECR_REPO
from util.metadata import (
LINUX_X86_ECR_REPO,
LINUX_AARCH_ECR_REPO,
WINDOWS_X86_ECR_REPO,
PIPELINE_ACCOUNT,
PIPELINE_REGION,
DEPLOY_ACCOUNT,
DEPLOY_REGION,
)

# Initialize app.
app = App()

# Initialize env.
env = Environment(account=AWS_ACCOUNT, region=AWS_REGION)

# Define AWS ECR stacks.
# ECR holds the docker images, which are pre-built to accelerate the code builds/tests of git pull requests.
EcrStack(app, "aws-lc-ecr-linux-x86", LINUX_X86_ECR_REPO, env=env)
EcrStack(app, "aws-lc-ecr-linux-aarch", LINUX_AARCH_ECR_REPO, env=env)
EcrStack(app, "aws-lc-ecr-windows-x86", WINDOWS_X86_ECR_REPO, env=env)

# Define CodeBuild Batch job for building Docker images.
LinuxDockerImageBatchBuildStack(app, "aws-lc-docker-image-build-linux", env=env)

# AWS CodeBuild cannot build Windows Docker images because DIND (Docker In Docker) is not supported on Windows.
# Windows Docker images are created by running commands in Windows EC2 instance.
WindowsDockerImageBuildStack(app, "aws-lc-docker-image-build-windows", env=env)

# Define CodeBuild Batch job for testing code.
x86_build_spec_file = "cdk/codebuild/github_ci_linux_x86_omnibus.yaml"
AwsLcGitHubCIStack(app, "aws-lc-ci-linux-x86", x86_build_spec_file, env=env)
arm_build_spec_file = "cdk/codebuild/github_ci_linux_arm_omnibus.yaml"
AwsLcGitHubCIStack(app, "aws-lc-ci-linux-arm", arm_build_spec_file, env=env)
integration_build_spec_file = "cdk/codebuild/github_ci_integration_omnibus.yaml"
AwsLcGitHubCIStack(app, "aws-lc-ci-integration", integration_build_spec_file, env=env)
win_x86_build_spec_file = "cdk/codebuild/github_ci_windows_x86_omnibus.yaml"
AwsLcGitHubCIStack(app, "aws-lc-ci-windows-x86", win_x86_build_spec_file, env=env)
fuzz_build_spec_file = "cdk/codebuild/github_ci_fuzzing_omnibus.yaml"
AwsLcGitHubFuzzCIStack(app, "aws-lc-ci-fuzzing", fuzz_build_spec_file, env=env)
analytics_build_spec_file = "cdk/codebuild/github_ci_analytics_omnibus.yaml"
AwsLcGitHubAnalyticsStack(app, "aws-lc-ci-analytics", analytics_build_spec_file, env=env)
# bm_framework_build_spec_file = "cdk/codebuild/bm_framework_omnibus.yaml"
# BmFrameworkStack(app, "aws-lc-ci-bm-framework", bm_framework_build_spec_file, env=env)
ec2_test_framework_build_spec_file = "cdk/codebuild/ec2_test_framework_omnibus.yaml"
AwsLcEC2TestingCIStack(app, "aws-lc-ci-ec2-test-framework", ec2_test_framework_build_spec_file, env=env)
android_build_spec_file = "cdk/codebuild/github_ci_android_omnibus.yaml"
AwsLcAndroidCIStack(app, "aws-lc-ci-devicefarm-android", android_build_spec_file, env=env)
AwsLcGitHubX509CIStack(app, "aws-lc-ci-x509")
AwsLcCiPipeline(
app,
"AwsLcCiPipeline",
env=Environment(account=PIPELINE_ACCOUNT, region=PIPELINE_REGION),
)

if DEPLOY_ACCOUNT and DEPLOY_REGION:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If these aren't set what happens when this calls app.synth?

# Initialize env.
env = Environment(account=DEPLOY_ACCOUNT, region=DEPLOY_REGION)

# Define AWS ECR stacks.
# ECR holds the docker images, which are pre-built to accelerate the code builds/tests of git pull requests.
EcrStack(app, "aws-lc-ecr-linux-x86", LINUX_X86_ECR_REPO, env=env)
EcrStack(app, "aws-lc-ecr-linux-aarch", LINUX_AARCH_ECR_REPO, env=env)
EcrStack(app, "aws-lc-ecr-windows-x86", WINDOWS_X86_ECR_REPO, env=env)

# Define CodeBuild Batch job for building Docker images.
LinuxDockerImageBatchBuildStack(app, "aws-lc-docker-image-build-linux", env=env)

# AWS CodeBuild cannot build Windows Docker images because DIND (Docker In Docker) is not supported on Windows.
# Windows Docker images are created by running commands in Windows EC2 instance.
WindowsDockerImageBuildStack(app, "aws-lc-docker-image-build-windows", env=env)

add_ci_stacks(app, env=env)

app.synth()
Loading
Loading