Skip to content

Commit c625306

Browse files
lzchenlzchen
committed
2 parents a69f01a + ed26bd0 commit c625306

File tree

176 files changed

+1650
-1111
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

176 files changed

+1650
-1111
lines changed

‎CODEOWNERS

+1-1
Original file line numberDiff line numberDiff line change
@@ -10,4 +10,4 @@
1010
# For all file changes, github would automatically
1111
# include the following people in the PRs.
1212

13-
* @vrdmr @gavin-aguiar @YunchuWang @pdthummar @hallvictoria
13+
* @vrdmr @gavin-aguiar @hallvictoria

‎README.md

+16-17
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,34 @@
11
# <img src="https://raw.githubusercontent.com/Azure/azure-functions-python-worker/dev/docs/Azure.Functions.svg" width = "30" alt="Functions Header Image - Lightning Logo"> Azure Functions Python Worker
22

3-
| Branch | Status | CodeCov | Unittests | E2E tests |
4-
|--------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------|
5-
| main | [![Build Status](https://azfunc.visualstudio.com/Azure%20Functions/_apis/build/status/Azure.azure-functions-python-worker?branchName=main)](https://azfunc.visualstudio.com/Azure%20Functions/_build/latest?definitionId=57&branchName=main) | [![codecov](https://codecov.io/gh/Azure/azure-functions-python-worker/branch/main/graph/badge.svg)](https://codecov.io/gh/Azure/azure-functions-python-worker) | ![CI Unit tests](https://github.com/Azure/azure-functions-python-worker/workflows/CI%20Unit%20tests/badge.svg?branch=main) | ![CI E2E tests](https://github.com/Azure/azure-functions-python-worker/workflows/CI%20E2E%20tests/badge.svg?branch=main) |
6-
| dev | [![Build Status](https://azfunc.visualstudio.com/Azure%20Functions/_apis/build/status/Azure.azure-functions-python-worker?branchName=dev)](https://azfunc.visualstudio.com/Azure%20Functions/_build/latest?definitionId=57&branchName=dev) | [![codecov](https://codecov.io/gh/Azure/azure-functions-python-worker/branch/dev/graph/badge.svg)](https://codecov.io/gh/Azure/azure-functions-python-worker) | ![CI Unit tests](https://github.com/Azure/azure-functions-python-worker/workflows/CI%20Unit%20tests/badge.svg?branch=dev) | ![CI E2E tests](https://github.com/Azure/azure-functions-python-worker/workflows/CI%20E2E%20tests/badge.svg?branch=dev) |
3+
| Branch | Build Status | CodeCov | Test Status |
4+
|--------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
5+
| dev | [![Build Status](https://img.shields.io/azure-devops/build/azfunc/public/658/dev)](https://azfunc.visualstudio.com/public/_build/latest?definitionId=658&branchName=dev) | [![codecov](https://codecov.io/gh/Azure/azure-functions-python-worker/branch/dev/graph/badge.svg)](https://codecov.io/gh/Azure/azure-functions-python-worker) | [![Test Status](https://img.shields.io/azure-devops/build/azfunc/public/658/dev)](https://azfunc.visualstudio.com/public/_build/latest?definitionId=658&branchName=dev) |
76

8-
Python support for Azure Functions is based on Python 3.6, 3.7, 3.8, 3.9, and 3.10 serverless hosting on Linux and the Functions 2.0, 3.0 and 4.0 runtime.
7+
Python support for Azure Functions is based on Python 3.8, 3.9, 3.10, 3.11, and 3.12 serverless hosting on Linux and the Functions 4.0 runtime.
98

109
Here is the current status of Python in Azure Functions:
1110

1211
What are the supported Python versions?
1312

14-
| Azure Functions Runtime | Python 3.6 | Python 3.7 | Python 3.8 | Python 3.9 | Python 3.10 | Python 3.11 |
15-
|----------------------------------|------------|------------|------------|------------|-------------|-------------|
16-
| Azure Functions 2.0 (deprecated) | ✔ | ✔ | - | - | - | - |
17-
| Azure Functions 3.0 (deprecated) | ✔ | ✔ | ✔ | ✔ | - | - |
18-
| Azure Functions 4.0 | - | - | ✔ | ✔ | ✔ | ✔ |
13+
| Azure Functions Runtime | Python 3.8 | Python 3.9 | Python 3.10 | Python 3.11 | Python 3.12 |
14+
|----------------------------------|------------|------------|-------------|-------------|-------------|
15+
| Azure Functions 3.0 (deprecated) | ✔ | ✔ | - | - | - |
16+
| Azure Functions 4.0 | ✔ | ✔ | ✔ | ✔ | ✔ |
1917

2018
For information about Azure Functions Runtime, please refer to [Azure Functions runtime versions overview](https://docs.microsoft.com/en-us/azure/azure-functions/functions-versions) page.
2119

2220
### What's available?
2321

24-
- Build, test, debug and publish using Azure Functions Core Tools (CLI) or Visual Studio Code
25-
- Deploy Python Function project onto consumption, dedicated, or elastic premium plan.
26-
- Deploy Python Function project in a custom docker image onto dedicated, or elastic premium plan.
27-
- Triggers / Bindings : HTTP, Blob, Queue, Timer, Cosmos DB, Event Grid, Event Hubs and Service Bus
22+
- Build, test, debug, and publish using Azure Functions Core Tools (CLI) or Visual Studio Code
23+
- Deploy Python Function project onto consumption, dedicated, elastic premium, or flex consumption plan.
24+
- Deploy Python Function project in a custom docker image onto dedicated or elastic premium plan.
25+
- Triggers / Bindings : Blob, Cosmos DB, Event Grid, Event Hub, HTTP, Kafka, MySQL, Queue, ServiceBus, SQL, Timer, and Warmup
2826
- Triggers / Bindings : Custom binding support
2927

30-
What's coming?
28+
### What's new?
3129

32-
- [Durable Functions For Python](https://github.com/Azure/azure-functions-durable-python)
30+
- [SDK Type Bindings for Blob](https://techcommunity.microsoft.com/t5/azure-compute-blog/azure-functions-sdk-type-bindings-for-azure-blob-storage-with/ba-p/4146744)
31+
- [HTTP Streaming](https://techcommunity.microsoft.com/t5/azure-compute-blog/azure-functions-support-for-http-streams-in-python-is-now-in/ba-p/4146697)
3332

3433
### Get Started
3534

@@ -72,4 +71,4 @@ provided by the bot. You will only need to do this once across all repos using o
7271

7372
This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/).
7473
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
75-
contact [[email protected]](mailto:[email protected]) with any additional questions or comments.
74+
contact [[email protected]](mailto:[email protected]) with any additional questions or comments.

‎azure_functions_worker/version.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
# Copyright (c) Microsoft Corporation. All rights reserved.
22
# Licensed under the MIT License.
33

4-
VERSION = '4.33.0'
4+
VERSION = '4.34.0'

‎eng/ci/emulator-tests.yml

+46
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
trigger: none # ensure this is not ran as a CI build
2+
3+
pr:
4+
branches:
5+
include:
6+
- dev
7+
- release/*
8+
9+
schedules:
10+
- cron: "0 8 * * 1,2,3,4,5"
11+
displayName: Monday to Friday 3 AM CST build
12+
branches:
13+
include:
14+
- dev
15+
always: true
16+
17+
resources:
18+
repositories:
19+
- repository: 1es
20+
type: git
21+
name: 1ESPipelineTemplates/1ESPipelineTemplates
22+
ref: refs/tags/release
23+
- repository: eng
24+
type: git
25+
name: engineering
26+
ref: refs/tags/release
27+
28+
variables:
29+
- template: /ci/variables/build.yml@eng
30+
- template: /ci/variables/cfs.yml@eng
31+
- template: /eng/templates/utils/variables.yml@self
32+
33+
extends:
34+
template: v1/1ES.Unofficial.PipelineTemplate.yml@1es
35+
parameters:
36+
pool:
37+
name: 1es-pool-azfunc
38+
image: 1es-windows-2022
39+
os: windows
40+
41+
stages:
42+
- stage: RunEmulatorTests
43+
jobs:
44+
- template: /eng/templates/jobs/ci-emulator-tests.yml@self
45+
parameters:
46+
PoolName: 1es-pool-azfunc

‎eng/ci/official-build.yml

+8
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,8 @@ resources:
3030
variables:
3131
- template: /eng/templates/utils/variables.yml@self
3232
- template: /eng/templates/utils/official-variables.yml@self
33+
- name: codeql.excludePathPatterns
34+
value: deps/,build/
3335

3436
extends:
3537
template: v1/1ES.Official.PipelineTemplate.yml@1es
@@ -52,6 +54,12 @@ extends:
5254
dependsOn: Build
5355
jobs:
5456
- template: /eng/templates/official/jobs/ci-e2e-tests.yml@self
57+
- stage: RunEmulatorTests
58+
dependsOn: Build
59+
jobs:
60+
- template: /eng/templates/jobs/ci-emulator-tests.yml@self
61+
parameters:
62+
PoolName: 1es-pool-azfunc
5563
- stage: RunUnitTests
5664
dependsOn: Build
5765
jobs:

‎eng/ci/public-build.yml

+15-6
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,14 @@ resources:
2828

2929
variables:
3030
- template: /eng/templates/utils/variables.yml@self
31+
- name: codeql.excludePathPatterns
32+
value: deps/,build/
33+
- name: codeql.compiled.enabled
34+
value: true
35+
- name: codeql.runSourceLanguagesInSourceAnalysis
36+
value: true
37+
- name: codeql.sourceLanguages
38+
value: python, powershell
3139

3240
extends:
3341
template: v1/1ES.Unofficial.PipelineTemplate.yml@1es
@@ -36,11 +44,6 @@ extends:
3644
name: 1es-pool-azfunc-public
3745
image: 1es-windows-2022
3846
os: windows
39-
sdl:
40-
codeql:
41-
compiled:
42-
enabled: true # still only runs for default branch
43-
runSourceLanguagesInSourceAnalysis: true
4447
settings:
4548
skipBuildTagsForGitHubPullRequests: ${{ variables['System.PullRequest.IsFork'] }}
4649
stages:
@@ -50,4 +53,10 @@ extends:
5053
- stage: RunUnitTests
5154
dependsOn: Build
5255
jobs:
53-
- template: /eng/templates/jobs/ci-unit-tests.yml@self
56+
- template: /eng/templates/jobs/ci-unit-tests.yml@self
57+
- stage: RunEmulatorTests
58+
dependsOn: Build
59+
jobs:
60+
- template: /eng/templates/jobs/ci-emulator-tests.yml@self
61+
parameters:
62+
PoolName: 1es-pool-azfunc-public

‎eng/templates/jobs/build.yml

+5-1
Original file line numberDiff line numberDiff line change
@@ -21,4 +21,8 @@ jobs:
2121
python -m pip install .
2222
displayName: 'Build python worker'
2323
# Skip the build stage for SDK and Extensions release branches. This stage will fail because pyproject.toml contains the updated (and unreleased) library version
24-
condition: and(eq(variables.isSdkRelease, false), eq(variables.isExtensionsRelease, false), eq(variables['USETESTPYTHONSDK'], false), eq(variables['USETESTPYTHONEXTENSIONS'], false))
24+
condition: and(eq(variables.isSdkRelease, false), eq(variables.isExtensionsRelease, false), eq(variables['USETESTPYTHONSDK'], false), eq(variables['USETESTPYTHONEXTENSIONS'], false))
25+
- bash: |
26+
pip install pip-audit
27+
pip-audit -r requirements.txt
28+
displayName: 'Run vulnerability scan'

‎eng/templates/jobs/ci-emulator-tests.yml

+100
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
jobs:
2+
- job: "TestPython"
3+
displayName: "Run Python Emulator Tests"
4+
5+
pool:
6+
name: ${{ parameters.PoolName }}
7+
image: 1es-ubuntu-22.04
8+
os: linux
9+
10+
strategy:
11+
matrix:
12+
Python37:
13+
PYTHON_VERSION: '3.7'
14+
Python38:
15+
PYTHON_VERSION: '3.8'
16+
Python39:
17+
PYTHON_VERSION: '3.9'
18+
Python310:
19+
PYTHON_VERSION: '3.10'
20+
Python311:
21+
PYTHON_VERSION: '3.11'
22+
Python312:
23+
PYTHON_VERSION: '3.12'
24+
steps:
25+
- task: UsePythonVersion@0
26+
inputs:
27+
versionSpec: $(PYTHON_VERSION)
28+
- task: UseDotNet@2
29+
displayName: 'Install .NET 8'
30+
inputs:
31+
version: 8.0.x
32+
- bash: |
33+
chmod +x eng/scripts/install-dependencies.sh
34+
chmod +x eng/scripts/test-setup.sh
35+
36+
eng/scripts/install-dependencies.sh $(PYTHON_VERSION)
37+
eng/scripts/test-setup.sh
38+
displayName: 'Install dependencies and the worker'
39+
condition: and(eq(variables.isSdkRelease, false), eq(variables.isExtensionsRelease, false), eq(variables['USETESTPYTHONSDK'], false), eq(variables['USETESTPYTHONEXTENSIONS'], false))
40+
- task: DownloadPipelineArtifact@2
41+
displayName: 'Download Python SDK Artifact'
42+
inputs:
43+
buildType: specific
44+
artifactName: 'azure-functions'
45+
project: 'internal'
46+
definition: 679
47+
buildVersionToDownload: latest
48+
targetPath: '$(Pipeline.Workspace)/PythonSdkArtifact'
49+
condition: or(eq(variables.isSdkRelease, true), eq(variables['USETESTPYTHONSDK'], true))
50+
- bash: |
51+
chmod +x eng/scripts/test-sdk.sh
52+
chmod +x eng/scripts/test-setup.sh
53+
54+
eng/scripts/test-sdk.sh $(Pipeline.Workspace) $(PYTHON_VERSION)
55+
eng/scripts/test-setup.sh
56+
displayName: 'Install test python sdk, dependencies and the worker'
57+
condition: or(eq(variables.isSdkRelease, true), eq(variables['USETESTPYTHONSDK'], true))
58+
- task: DownloadPipelineArtifact@2
59+
displayName: 'Download Python Extension Artifact'
60+
inputs:
61+
buildType: specific
62+
artifactName: $(PYTHONEXTENSIONNAME)
63+
project: 'internal'
64+
definition: 798
65+
buildVersionToDownload: latest
66+
targetPath: '$(Pipeline.Workspace)/PythonExtensionArtifact'
67+
condition: or(eq(variables.isExtensionsRelease, true), eq(variables['USETESTPYTHONEXTENSIONS'], true))
68+
- bash: |
69+
chmod +x eng/scripts/test-setup.sh
70+
chmod +x eng/scripts/test-extensions.sh
71+
72+
eng/scripts/test-extensions.sh $(Pipeline.Workspace) $(PYTHON_VERSION)
73+
eng/scripts/test-setup.sh
74+
displayName: 'Install test python extension, dependencies and the worker'
75+
condition: or(eq(variables.isExtensionsRelease, true), eq(variables['USETESTPYTHONEXTENSIONS'], true))
76+
- bash: |
77+
docker compose -f tests/emulator_tests/utils/eventhub/docker-compose.yml pull
78+
docker compose -f tests/emulator_tests/utils/eventhub/docker-compose.yml up -d
79+
displayName: 'Install Azurite and Start EventHub Emulator'
80+
- bash: |
81+
python -m pytest -q -n auto --dist loadfile --reruns 4 --ignore=tests/emulator_tests/test_servicebus_functions.py tests/emulator_tests
82+
env:
83+
AzureWebJobsStorage: "UseDevelopmentStorage=true"
84+
AzureWebJobsEventHubConnectionString: "Endpoint=sb://localhost;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SAS_KEY_VALUE;UseDevelopmentEmulator=true;"
85+
displayName: "Running $(PYTHON_VERSION) Python Linux Emulator Tests"
86+
- bash: |
87+
# Stop and remove EventHub Emulator container to free up the port
88+
docker stop eventhubs-emulator
89+
docker container rm --force eventhubs-emulator
90+
docker compose -f tests/emulator_tests/utils/servicebus/docker-compose.yml pull
91+
docker compose -f tests/emulator_tests/utils/servicebus/docker-compose.yml up -d
92+
env:
93+
AzureWebJobsSQLPassword: $(AzureWebJobsSQLPassword)
94+
displayName: 'Install Azurite and Start ServiceBus Emulator'
95+
- bash: |
96+
python -m pytest -q -n auto --dist loadfile --reruns 4 tests/emulator_tests/test_servicebus_functions.py
97+
env:
98+
AzureWebJobsStorage: "UseDevelopmentStorage=true"
99+
AzureWebJobsServiceBusConnectionString: "Endpoint=sb://localhost;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SAS_KEY_VALUE;UseDevelopmentEmulator=true;"
100+
displayName: "Running $(PYTHON_VERSION) Python ServiceBus Linux Emulator Tests"

‎eng/templates/official/jobs/ci-e2e-tests.yml

+20-1
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,25 @@ jobs:
115115
eng/scripts/test-setup.sh
116116
displayName: 'Install test python extension, dependencies and the worker'
117117
condition: or(eq(variables.isExtensionsRelease, true), eq(variables['USETESTPYTHONEXTENSIONS'], true))
118+
- powershell: |
119+
$pipelineVarSet = "$(USETESTPYTHONSDK)"
120+
Write-Host "pipelineVarSet: $pipelineVarSet"
121+
$branch = "$(Build.SourceBranch)"
122+
Write-Host "Branch: $branch"
123+
if($branch.StartsWith("refs/heads/sdk/") -or $pipelineVarSet -eq "true")
124+
{
125+
Write-Host "##vso[task.setvariable variable=skipTest;]true"
126+
}
127+
else
128+
{
129+
Write-Host "##vso[task.setvariable variable=skipTest;]false"
130+
}
131+
displayName: 'Set skipTest variable'
132+
condition: or(eq(variables.isSdkRelease, true), eq(variables['USETESTPYTHONSDK'], true))
133+
- powershell: |
134+
Write-Host "skipTest: $(skipTest)"
135+
displayName: 'Display skipTest variable'
136+
condition: or(eq(variables.isSdkRelease, true), eq(variables['USETESTPYTHONSDK'], true))
118137
- bash: |
119138
python -m pytest -q -n auto --dist loadfile --reruns 4 --cov=./azure_functions_worker --cov-report xml --cov-branch --cov-append tests/endtoend tests/extension_tests/deferred_bindings_tests tests/extension_tests/http_v2_tests
120139
env:
@@ -125,5 +144,5 @@ jobs:
125144
AzureWebJobsSqlConnectionString: $(SQL_CONNECTION)
126145
AzureWebJobsEventGridTopicUri: $(EVENTGRID_URI)
127146
AzureWebJobsEventGridConnectionKey: $(EVENTGRID_CONNECTION)
128-
USETESTPYTHONSDK: or(eq(variables.isSdkRelease, true), eq(variables['USETESTPYTHONSDK'], true))
147+
skipTest: $(skipTest)
129148
displayName: "Running $(PYTHON_VERSION) Python E2E Tests"

0 commit comments

Comments
 (0)