Python: Add Integration tests for AzureFunctions #2020
Conversation
markwallace-microsoft
added
documentation
Python Test Coverage Report •
Python Unit Test Overview
|
There was a problem hiding this comment.
Pull Request Overview
This PR adds comprehensive integration tests for Azure Functions samples in the Durable Agent Framework. The tests automatically start and stop Azure Functions apps for each sample using dynamically allocated ports, and validate their behavior through HTTP calls.
- Adds integration test infrastructure with fixtures for managing function app lifecycle
- Implements test coverage for 7 Azure Functions samples (single agent, multi-agent, orchestration patterns, callbacks, and HITL)
- Updates CI workflow to set up required emulators (Azurite and Durable Task Scheduler)
Reviewed Changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
python/packages/azurefunctions/tests/integration_tests/testutils.py |
Core test utilities for managing function app lifecycle, port allocation, and orchestration polling |
python/packages/azurefunctions/tests/integration_tests/conftest.py |
Pytest configuration with fixtures for starting/stopping function apps based on markers |
python/packages/azurefunctions/tests/integration_tests/test_*.py |
Integration tests for each sample (01-07), validating HTTP endpoints and orchestration behavior |
python/packages/azurefunctions/tests/integration_tests/README.md |
Documentation for running integration tests and required setup |
.github/actions/azure-functions-integration-setup/action.yml |
GitHub Actions setup for emulators (Azurite and DTS) |
.github/workflows/python-merge-tests.yml |
Updated CI workflow to include emulator setup |
python/samples/getting_started/azure_functions/07_single_agent_orchestration_hitl/function_app.py |
Simplified type hints from dict[str, Any] to dict |
python/packages/azurefunctions/tests/integration_tests/README.md
Outdated
Show resolved
Hide resolved
python/packages/azurefunctions/tests/integration_tests/README.md
Outdated
Show resolved
Hide resolved
Co-authored-by: Copilot <[email protected]>
...n/samples/getting_started/azure_functions/07_single_agent_orchestration_hitl/function_app.py
Show resolved
Hide resolved
...n/samples/getting_started/azure_functions/07_single_agent_orchestration_hitl/function_app.py
Show resolved
Hide resolved
python/samples/getting_started/azure_functions/07_single_agent_orchestration_hitl/README.md
Outdated
Show resolved
Hide resolved
Co-authored-by: Copilot <[email protected]>
|
|
||
| # Azure Functions Configuration | ||
| AzureWebJobsStorage=UseDevelopmentStorage=true | ||
| DURABLE_TASK_SCHEDULER_CONNECTION_STRING=Endpoint=http://localhost:8080;Authentication=None |
There was a problem hiding this comment.
The .env.example file lists TASKHUB_NAME as not required for integration tests, but TASKHUB_NAME is set dynamically in start_function_app() in testutils.py (line 301). However, the README.md documentation (line 21) does not mention TASKHUB_NAME as a required variable, which is correct since it's generated dynamically. Consider adding a comment in the .env.example explaining that TASKHUB_NAME is not needed for integration tests as it's auto-generated per test run.
| DURABLE_TASK_SCHEDULER_CONNECTION_STRING=Endpoint=http://localhost:8080;Authentication=None | |
| DURABLE_TASK_SCHEDULER_CONNECTION_STRING=Endpoint=http://localhost:8080;Authentication=None | |
| # Note: TASKHUB_NAME is not required for integration tests; it is auto-generated per test run. |
| sample_path, error_message = get_sample_path_from_marker(request) | ||
| if error_message: | ||
| pytest.fail(error_message) | ||
| return |
There was a problem hiding this comment.
The return statement on line 79 after pytest.fail() is unreachable because pytest.fail() raises an exception. This line should be removed as it serves no purpose and creates dead code.
| return |
| sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) | ||
| result = sock.connect_ex((host, port)) == 0 | ||
| sock.close() |
There was a problem hiding this comment.
The is_port_in_use() function does not close the socket in all code paths. If sock.connect_ex() raises an exception (though unlikely with connect_ex), the socket won't be closed. Consider using a context manager or try-finally to ensure the socket is always closed.
| sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) | |
| result = sock.connect_ex((host, port)) == 0 | |
| sock.close() | |
| with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock: | |
| result = sock.connect_ex((host, port)) == 0 |
| if [ "$(docker ps -aq -f name=azurite)" ]; then | ||
| docker rm -f azurite | ||
| fi | ||
| docker run -d --name azurite -p 10000:10000 -p 10001:10001 -p 10002:10002 mcr.microsoft.com/azure-storage/azurite |
There was a problem hiding this comment.
After starting Azurite on line 21, there's no health check to ensure it's ready before proceeding. Unlike the DTS emulator (lines 13-14) which waits for a health endpoint, Azurite is started without verification. Consider adding a health check or sleep to ensure Azurite is ready before the tests begin, similar to the DTS emulator setup.
| docker run -d --name azurite -p 10000:10000 -p 10001:10001 -p 10002:10002 mcr.microsoft.com/azure-storage/azurite | |
| docker run -d --name azurite -p 10000:10000 -p 10001:10001 -p 10002:10002 mcr.microsoft.com/azure-storage/azurite | |
| # Wait for Azurite to be ready by polling the blob service endpoint | |
| timeout 30 bash -c 'until curl --silent --fail http://localhost:10000/devstoreaccount1?comp=list; do sleep 1; done' |
| - name: Set up Azure Functions Integration Test Emulators | ||
| uses: ./.github/actions/azure-functions-integration-setup | ||
| id: azure-functions-setup |
There was a problem hiding this comment.
[nitpick] The Azure Functions Integration Test Setup action is being added, but there's no environment variable passed to the tests to indicate that these emulators are available. Consider verifying that the integration tests can properly discover and connect to the DTS emulator at http://localhost:8080 and Azurite at the default ports without additional configuration.
| sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) | ||
| result = sock.connect_ex((host, port)) == 0 | ||
| sock.close() | ||
| return result |
There was a problem hiding this comment.
The socket resource should use a context manager to ensure proper cleanup even if an exception occurs. Replace with: with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:\n return sock.connect_ex((host, port)) == 0
| sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) | |
| result = sock.connect_ex((host, port)) == 0 | |
| sock.close() | |
| return result | |
| with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock: | |
| return sock.connect_ex((host, port)) == 0 |
3 participants
Motivation and Context
Add integration tests for Azure Functions.
Description
The tests depend on our samples to provide the structure and our tests validate that they're working.
Azure Functions workers rely on the Functions host and a bunch of other emulators (azurite, DTS) to successfully run locally. This PR adds the setup for those emulators and functions host and starts it up before testing.
Contribution Checklist