Skip to content

Commit 398ae5a

Browse files
pdecatantonbabenko
pdecat
and
antonbabenko
authored
feat: Support installing poetry dependencies with pip (#311)
Co-authored-by: Anton Babenko <[email protected]>
1 parent c231754 commit 398ae5a

File tree

17 files changed

+436
-12
lines changed

17 files changed

+436
-12
lines changed

‎.github/workflows/test.yml

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
name: Tests
2+
3+
env:
4+
PYTEST_VERSION: 7.1.3
5+
6+
on:
7+
push:
8+
branches: [master]
9+
tags: ["*"]
10+
pull_request:
11+
branches: [master]
12+
13+
jobs:
14+
tests:
15+
name: Test with Python ${{ matrix.python_version }}
16+
runs-on: ubuntu-latest
17+
strategy:
18+
matrix:
19+
python_version: ["3.7", "3.8", "3.9", "3.10"]
20+
fail-fast: false
21+
steps:
22+
- uses: actions/checkout@v3
23+
24+
- name: Set up Python ${{ matrix.python_version }}
25+
uses: actions/setup-python@v3
26+
with:
27+
python-version: ${{ matrix.python_version }}
28+
29+
- name: Install poetry and tox
30+
shell: bash
31+
run: |
32+
pip install pytest==${PYTEST_VERSION}
33+
34+
- name: Run tox
35+
shell: bash
36+
run: |
37+
python -m pytest -vvv tests/

‎.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,3 +32,6 @@ terraform.rc
3232
# Lambda directories
3333
builds/
3434
__pycache__/
35+
36+
# Test directories
37+
.tox

‎README.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -819,6 +819,29 @@ No modules.
819819
| <a name="output_s3_object"></a> [s3\_object](#output\_s3\_object) | The map with S3 object data of zip archive deployed (if deployment was from S3) |
820820
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
821821

822+
## Development
823+
824+
### Python
825+
826+
During development involving modifying python files, use tox to run unit tests:
827+
828+
```
829+
tox
830+
```
831+
832+
This will try to run unit tests which each supported python version, reporting errors for python versions which are not installed locally.
833+
834+
If you only want to test against your main python version:
835+
836+
```
837+
tox -e py
838+
```
839+
840+
You can also pass additional positional arguments to pytest which is used to run test, e.g. to make it verbose:
841+
```
842+
tox -e py -- -vvv
843+
```
844+
822845
## Authors
823846

824847
Module managed by [Anton Babenko](https://github.com/antonbabenko). Check out [serverless.tf](https://serverless.tf) to learn more about doing serverless with Terraform.

‎examples/build-package/README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,8 +36,10 @@ Note that this example may create resources which cost money. Run `terraform des
3636
| <a name="module_lambda_function_from_package"></a> [lambda\_function\_from\_package](#module\_lambda\_function\_from\_package) | ../../ | n/a |
3737
| <a name="module_lambda_layer"></a> [lambda\_layer](#module\_lambda\_layer) | ../../ | n/a |
3838
| <a name="module_lambda_layer_pip_requirements"></a> [lambda\_layer\_pip\_requirements](#module\_lambda\_layer\_pip\_requirements) | ../.. | n/a |
39+
| <a name="module_lambda_layer_poetry"></a> [lambda\_layer\_poetry](#module\_lambda\_layer\_poetry) | ../../ | n/a |
3940
| <a name="module_package_dir"></a> [package\_dir](#module\_package\_dir) | ../../ | n/a |
4041
| <a name="module_package_dir_pip_dir"></a> [package\_dir\_pip\_dir](#module\_package\_dir\_pip\_dir) | ../../ | n/a |
42+
| <a name="module_package_dir_poetry"></a> [package\_dir\_poetry](#module\_package\_dir\_poetry) | ../../ | n/a |
4143
| <a name="module_package_dir_with_npm_install"></a> [package\_dir\_with\_npm\_install](#module\_package\_dir\_with\_npm\_install) | ../../ | n/a |
4244
| <a name="module_package_dir_without_npm_install"></a> [package\_dir\_without\_npm\_install](#module\_package\_dir\_without\_npm\_install) | ../../ | n/a |
4345
| <a name="module_package_dir_without_pip_install"></a> [package\_dir\_without\_pip\_install](#module\_package\_dir\_without\_pip\_install) | ../../ | n/a |

‎examples/build-package/main.tf

Lines changed: 52 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -17,28 +17,52 @@ resource "random_pet" "this" {
1717
# Build packages
1818
#################
1919

20-
# Create zip-archive of a single directory where "pip install" will also be executed (default for python runtime)
20+
# Create zip-archive of a single directory where "pip install" will also be executed (default for python runtime with requirements.txt present)
2121
module "package_dir" {
2222
source = "../../"
2323

2424
create_function = false
2525

26-
runtime = "python3.8"
27-
source_path = "${path.module}/../fixtures/python3.8-app1"
26+
build_in_docker = true
27+
runtime = "python3.8"
28+
source_path = "${path.module}/../fixtures/python3.8-app1"
29+
artifacts_dir = "${path.root}/builds/package_dir/"
2830
}
2931

30-
# Create zip-archive of a single directory where "pip install" will also be executed (default for python runtime) and set temporary directory for pip install
32+
# Create zip-archive of a single directory where "pip install" will also be executed (default for python runtime with requirements.txt present) and set temporary directory for pip install
3133
module "package_dir_pip_dir" {
3234
source = "../../"
3335

3436
create_function = false
3537

36-
runtime = "python3.8"
38+
build_in_docker = true
39+
runtime = "python3.8"
3740
source_path = [{
3841
path = "${path.module}/../fixtures/python3.8-app1"
3942
pip_tmp_dir = "${path.cwd}/../fixtures"
4043
pip_requirements = "${path.module}/../fixtures/python3.8-app1/requirements.txt"
4144
}]
45+
artifacts_dir = "${path.root}/builds/package_dir_pip_dir/"
46+
}
47+
48+
# Create zip-archive of a single directory where "poetry export" & "pip install --no-deps" will also be executed
49+
module "package_dir_poetry" {
50+
source = "../../"
51+
52+
create_function = false
53+
54+
build_in_docker = true
55+
runtime = "python3.9"
56+
docker_image = "build-python3.9-poetry"
57+
docker_file = "${path.module}/../fixtures/python3.9-app-poetry/docker/Dockerfile"
58+
59+
source_path = [
60+
{
61+
path = "${path.module}/../fixtures/python3.9-app-poetry"
62+
poetry_install = true
63+
}
64+
]
65+
artifacts_dir = "${path.root}/builds/package_dir_poetry/"
4266
}
4367

4468
# Create zip-archive of a single directory without running "pip install" (which is default for python runtime)
@@ -280,6 +304,29 @@ module "lambda_layer" {
280304
runtime = "python3.8"
281305
docker_image = "public.ecr.aws/sam/build-python3.8:latest"
282306
docker_file = "${path.module}/../fixtures/python3.8-app1/docker/Dockerfile"
307+
artifacts_dir = "${path.root}/builds/lambda_layer/"
308+
}
309+
310+
module "lambda_layer_poetry" {
311+
source = "../../"
312+
313+
create_layer = true
314+
layer_name = "${random_pet.this.id}-layer-poetry-dockerfile"
315+
compatible_runtimes = ["python3.9"]
316+
317+
source_path = [
318+
{
319+
path = "${path.module}/../fixtures/python3.9-app-poetry"
320+
poetry_install = true
321+
}
322+
]
323+
hash_extra = "extra-hash-to-prevent-conflicts-with-module.package_dir"
324+
325+
build_in_docker = true
326+
runtime = "python3.9"
327+
docker_image = "build-python3.9-poetry"
328+
docker_file = "${path.module}/../fixtures/python3.9-app-poetry/docker/Dockerfile"
329+
artifacts_dir = "${path.root}/builds/lambda_layer_poetry/"
283330
}
284331

285332
#######################

‎examples/fixtures/python3.8-app1/docker/Dockerfile

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
FROM lambci/lambda:build-python3.8 as build
1+
FROM public.ecr.aws/sam/build-python3.8 as build
22

33
LABEL maintainer="Betajob AS" \
44
description="Patched AWS Lambda build container"
@@ -20,7 +20,7 @@ RUN \
2020
&& rpmbuild -ba SPECS/automake.spec --nocheck \
2121
&& yum install -y RPMS/noarch/*
2222

23-
FROM lambci/lambda:build-python3.8
23+
FROM public.ecr.aws/sam/build-python3.8
2424
COPY --from=build /root/rpmbuild/RPMS/noarch/*.rpm .
2525
RUN yum install -y *.rpm \
2626
&& rm *.rpm

‎examples/fixtures/python3.9-app-poetry/docker/Dockerfile

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
FROM public.ecr.aws/sam/build-python3.9
2+
3+
LABEL maintainer="Betajob AS" \
4+
description="Patched AWS Lambda build container"
5+
6+
RUN pip install poetry==1.2.2

‎examples/fixtures/python3.9-app-poetry/ignore_please.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
This file should not be included in archive.

‎examples/fixtures/python3.9-app-poetry/index.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
def lambda_handler(event, context):
2+
print("Hello from app1!")
3+
4+
return event

‎examples/fixtures/python3.9-app-poetry/poetry.lock

Lines changed: 33 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎examples/fixtures/python3.9-app-poetry/poetry.toml

Whitespace-only changes.

‎examples/fixtures/python3.9-app-poetry/pyproject.toml

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
[tool.poetry]
2+
name = "python3.9-app-poetry"
3+
version = "0.1.0"
4+
description = ""
5+
authors = ["Your Name <[email protected]>"]
6+
7+
[tool.poetry.dependencies]
8+
python = "^3.7"
9+
colorful = "^0.5.4"
10+
11+
[tool.poetry.dev-dependencies]
12+
13+
[build-system]
14+
requires = ["poetry-core>=1.0.0"]
15+
build-backend = "poetry.core.masonry.api"

‎examples/simple/main.tf

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -301,7 +301,7 @@ module "lambda_function" {
301301
# docker_with_ssh_agent = true
302302
# docker_file = "${path.module}/../fixtures/python3.8-app1/docker/Dockerfile"
303303
# docker_build_root = "${path.module}/../../docker"
304-
# docker_image = "lambci/lambda:build-python3.8"
304+
# docker_image = "public.ecr.aws/sam/build-python3.8"
305305
}
306306

307307
####

0 commit comments

Comments
 (0)