Andystaples/add functions support #75

andystaples · 2025-11-21T20:48:06Z

DRAFT PR - for collaboration
Requires at minimum the changes here
Azure/azure-functions-durable-extension#3260
in durable WebJobs extension allowing gRPC protocol for Python

Copilot

Pull request overview

This draft PR introduces Azure Functions support for the durabletask-python library by creating a new durabletask-azurefunctions package. This allows developers to use Durable Task patterns within Azure Functions using Python decorators and bindings that integrate with the Azure Functions worker.

Key Changes:

Added a new durabletask-azurefunctions package with decorators, client, and worker implementations for Azure Functions integration
Modified core durabletask/worker.py to support a new ProtoTaskHubSidecarServiceStub type alongside the existing gRPC stub
Introduced a base ProtoTaskHubSidecarServiceStub class that can be extended for different communication patterns

Reviewed changes

Copilot reviewed 12 out of 15 changed files in this pull request and generated 22 comments.

Show a summary per file

File	Description
`durabletask/worker.py`	Added import for new stub type, updated type hints to accept Union of stub types, added handling for `orchestratorCompleted` event
`durabletask/internal/ProtoTaskHubSidecarServiceStub.py`	New base stub class defining the protocol interface with callable attributes for all Task Hub operations
`durabletask-azurefunctions/pyproject.toml`	Package configuration for the new Azure Functions integration package with dependencies
`durabletask-azurefunctions/durabletask/azurefunctions/worker.py`	Worker implementation that extends `TaskHubGrpcWorker` without async worker loop for Functions execution model
`durabletask-azurefunctions/durabletask/azurefunctions/client.py`	Client implementation for Azure Functions that parses connection info from JSON and uses custom interceptors
`durabletask-azurefunctions/durabletask/azurefunctions/internal/azurefunctions_null_stub.py`	Null stub implementation that provides no-op lambdas for all stub operations
`durabletask-azurefunctions/durabletask/azurefunctions/internal/azurefunctions_grpc_interceptor.py`	Custom gRPC interceptor that adds Azure Functions-specific headers
`durabletask-azurefunctions/durabletask/azurefunctions/decorators/metadata.py`	Trigger and binding metadata classes for orchestration, activity, entity, and client bindings
`durabletask-azurefunctions/durabletask/azurefunctions/decorators/durable_app.py`	Blueprint and DFApp classes providing decorators for registering Functions with Durable Task patterns
`durabletask-azurefunctions/durabletask/azurefunctions/decorators/__init__.py`	Package exports for decorator module
`durabletask-azurefunctions/durabletask/azurefunctions/constants.py`	Constants for trigger and binding type names
`durabletask-azurefunctions/CHANGELOG.md`	Initial changelog for the new package

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

durabletask-azurefunctions/durabletask/azurefunctions/internal/azurefunctions_null_stub.py

durabletask-azurefunctions/durabletask/azurefunctions/constants.py

durabletask-azurefunctions/durabletask/azurefunctions/worker.py

Copilot · 2025-11-21T20:55:36Z

durabletask-azurefunctions/durabletask/azurefunctions/client.py

+    creationUrls: dict[str, str]
+    managementUrls: dict[str, str]


[nitpick] Potential compatibility issue with type hint syntax. The use of dict[str, str] (PEP 585 style) requires Python 3.9+. While pyproject.toml specifies requires-python = ">=3.9", consider whether this is the intended minimum version or if Dict[str, str] from typing should be used for broader compatibility.

...letask-azurefunctions/durabletask/azurefunctions/internal/azurefunctions_grpc_interceptor.py

durabletask-azurefunctions/durabletask/azurefunctions/worker.py

durabletask/internal/ProtoTaskHubSidecarServiceStub.py

durabletask-azurefunctions/durabletask/azurefunctions/internal/azurefunctions_null_stub.py

durabletask-azurefunctions/durabletask/azurefunctions/worker.py

andystaples · 2025-11-21T20:49:40Z

durabletask-azurefunctions/durabletask/azurefunctions/decorators/durable_app.py

+                if response is None:
+                    raise Exception("Orchestrator execution did not produce a response.")
+                # The Python worker returns the input as type "json", so double-encoding is necessary
+                return '"' + base64.b64encode(response.SerializeToString()).decode('utf-8') + '"'


Victoria - Currently, the return value from here is passed on to the host as type "json" so the host attempts to Newtonsoft deserialize it back into an object before handing back to the Durable middleware for final decoding. This breaks, unless I double-encode with quotes as above. Is there a way to communicate to the worker that this is a plain string instead?

Investigating this - will need a little more time to test on my end

This is actually coming from the OrchestrationTriggerConverter done here. The type is hard-coded to json. Changing the type to string works in this scenario, but I'm unsure if there are other cases where json was intended

@classmethod def encode(cls, obj: typing.Any, *, expected_type: typing.Optional[type]) -> meta.Datum: # Durable function context should be a json return meta.Datum(type='json', value=obj)

andystaples · 2025-11-21T21:00:29Z

durabletask-azurefunctions/durabletask/azurefunctions/decorators/durable_app.py

+        # Obtain user-code and force type annotation on the client-binding parameter to be `str`.
+        # This ensures a passing type-check of that specific parameter,
+        # circumventing a limitation of the worker in type-checking rich DF Client objects.
+        # TODO: Once rich-binding type checking is possible, remove the annotation change.
+        user_code = fb._function._func
+        user_code.__annotations__[parameter_name] = str


Victoria - this is the same approach taken by the existing Durable Python SDK for the DurableClient binding - we force the annotation to be "str" so the worker takes a path that does not attempt to use the DurableClientConverter input parameter converter, which would throw NotImplementedError

https://github.com/Azure/azure-functions-python-library/blob/8bd30d300529cbda0526aa2d6606d11333c9c3aa/azure/functions/durable_functions.py#L130

Do you think it is worth moving the client_constructor logic in this PR into the DurableClientConverter in the -library, so that we don't have to do this type-hacking stuff?

We'd have to figure out how to detect which underlying provider for the durable_client_input binding is being used to know when to simply return the string for the old SDK vs parse it in the new

The main issue would be that we'd have something different to return based on the durable library.

Are the types going to be the same? (eg DurableClient for both packages) We could look at creating two separate converters - right now it's using the Generic converter, but it would be better to have our own

andystaples · 2025-11-21T21:02:15Z

durabletask-azurefunctions/pyproject.toml

+requires-python = ">=3.9"
+license = {file = "LICENSE"}
+readme = "README.md"
+dependencies = [
+    "durabletask>=0.5.0",
+    "azure-identity>=1.19.0",
+    "azure-functions>=1.11.0"
+]


TODO: Update python min version and rev durabletask dependency to 1.0.1/1.1.0
Also rev durabletask versions to the same based on size of changes needed

andystaples · 2025-11-21T21:02:40Z

durabletask-azurefunctions/pyproject.toml

+
+[project]
+name = "durabletask.azurefunctions"
+version = "0.1.0"


Victoria - what versioning strategy would you propose if the first version that goes to PyPi would be used for internal testing only?

dev versions work well - eg 1.0.0dev0 or 0.0.1dev0

andystaples · 2025-11-21T21:05:37Z

durabletask-azurefunctions/durabletask/azurefunctions/decorators/durable_app.py

+        # TODO: Is there a better way to support retrieving the unwrapped user code?
+        df_client_middleware.client_function = fb._function._func  # type: ignore


Victoria - not sure if you remember this context from a while back, but this is also carryover from the previous SDK - I added this line to make retrieving the "unwrapped" user code possible for the unit testing scenario - see
https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-unit-testing-python#unit-testing-trigger-functions
If possible, I'd like to see a "better" solution for the new SDK. Hate to re-open a can of worms here, though

I vaguely remember context, but we can sync again over specific requirements.

andystaples · 2025-11-21T21:07:14Z

durabletask-azurefunctions/durabletask/azurefunctions/decorators/durable_app.py

+                stub = AzureFunctionsNullStub()
+                worker = DurableFunctionsWorker()
+                response: Optional[OrchestratorResponse] = None
+
+                def stub_complete(stub_response):
+                    nonlocal response
+                    response = stub_response
+                stub.CompleteOrchestratorTask = stub_complete


All of this is probably optimizable - do we really need to create a new stub and worker for each call? Can they be saved? Will look into this more at some point

berndverst · 2025-11-24T18:49:04Z

Can you add an overview / more detail to the PR description?

- Still needs eventSent and eventRecieved implementations

- Add new_uuid method to OrchestrationContext for deterministic replay-safe UUIDs - Fix entity locking behavior for Functions - Align _RuntimeOrchestrationContext param names with OrchestrationContext - Remap __init__.py files for new module - Update version to 0.0.1dev0 - Add docstrings to missing methods - Move code for executing orchestrators/entities to DurableFunctionsWorker - Add function metadata to triggers for detection by extension

.github/workflows/durabletask-azurefunctions.yml

+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Python 3.14
+      uses: actions/setup-python@v5
+      with:
+        python-version: 3.14
+    - name: Install dependencies
+      working-directory: durabletask-azurefunctions
+      run: |
+        python -m pip install --upgrade pip
+        pip install setuptools wheel tox
+        pip install flake8
+    - name: Run flake8 Linter
+      working-directory: durabletask-azurefunctions
+      run: flake8 .
+    - name: Run flake8 Linter
+      working-directory: tests/durabletask-azurefunctions
+      run: flake8 .
+
+  run-docker-tests:


To address the problem, add an explicit permissions: block to the workflow to restrict the default permissions of the GITHUB_TOKEN.
The best and most secure method without altering existing workflow functionality is to add a top-level permissions: block with contents: read (sufficient for code checkout and most workflows), unless specific jobs require more, in which case you can override at the job level.

The changes should be made near the top of .github/workflows/durabletask-azurefunctions.yml, after the name: declaration and before on: (or immediately after, before the jobs section). No additional methods, external dependencies, or imports are needed.

.github/workflows/durabletask-azurefunctions.yml

+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
+    env:
+      EMULATOR_VERSION: "latest"
+    needs: lint
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Pull Docker image
+        run: docker pull mcr.microsoft.com/dts/dts-emulator:$EMULATOR_VERSION
+
+      - name: Run Docker container
+        run: |
+          docker run --name dtsemulator -d -p 8080:8080 mcr.microsoft.com/dts/dts-emulator:$EMULATOR_VERSION
+
+      - name: Wait for container to be ready
+        run: sleep 10  # Adjust if your service needs more time to start
+
+      - name: Set environment variables
+        run: |
+          echo "TASKHUB=default" >> $GITHUB_ENV
+          echo "ENDPOINT=http://localhost:8080" >> $GITHUB_ENV
+
+      - name: Install durabletask dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install flake8 pytest
+          pip install -r requirements.txt
+
+      - name: Install durabletask-azurefunctions dependencies
+        working-directory: examples
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements.txt
+
+      - name: Install durabletask-azurefunctions locally
+        working-directory: durabletask-azurefunctions
+        run: |
+          pip install . --no-deps --force-reinstall
+
+      - name: Install durabletask locally
+        run: |
+          pip install . --no-deps --force-reinstall
+
+      - name: Run the tests
+        working-directory: tests/durabletask-azurefunctions
+        run: |
+          pytest -m "dts" --verbose
+
+  publish:


To fix the problem, add an explicit permissions: block at the root of the workflow file (.github/workflows/durabletask-azurefunctions.yml). This will ensure all jobs use only the permissions required, and the default GITHUB_TOKEN permissions are restricted to the least privilege needed. For most workflows, setting contents: read is sufficient unless a job requires additional permissions. In this workflow, the publish job handles package publishing to PyPI using a secret and does not require contents: write, unless it also pushes tags or modifies repository content (which it does not shown based on the steps). Thus, the minimal safe change is to add a root-level permissions: contents: read entry near the top of the YAML file (typically right after name: and before on:).

.github/workflows/durabletask-azurefunctions.yml

+    if: startsWith(github.ref, 'refs/tags/azurefunctions-v')  # Only run if a matching tag is pushed
+    needs: run-docker-tests
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Extract version from tag
+        run: echo "VERSION=${GITHUB_REF#refs/tags/azurefunctions-v}" >> $GITHUB_ENV  # Extract version from the tag
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.14"  # Adjust Python version as needed
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install build twine
+
+      - name: Build package from directory durabletask-azurefunctions
+        working-directory: durabletask-azurefunctions
+        run: |
+          python -m build
+
+      - name: Check package
+        working-directory: durabletask-azurefunctions
+        run: |
+          twine check dist/* 
+
+      - name: Publish package to PyPI
+        env:
+          TWINE_USERNAME: __token__
+          TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN_AZUREFUNCTIONS  }}  # Store your PyPI API token in GitHub Secrets
+        working-directory: durabletask-azurefunctions
+        run: | 
+          twine upload dist/* 


To fix this issue, you should explicitly define a minimal permissions block in your workflow. Ideally, the workflow should declare permissions at the root level so all jobs default to least privilege unless they override this setting. The recommended setting for most workflows, particularly for CI and publishing to package registries, is contents: read, which allows jobs to read repository contents (e.g., for checkout), but not write or modify anything in the repository. Therefore, you should add:

permissions: contents: read

to the root of .github/workflows/durabletask-azurefunctions.yml, e.g., directly below the name: and before the on: block, or after the on: block but before jobs:. This will apply to all jobs in the workflow.

No additional imports, definitions, methods, or changes elsewhere are needed.

andystaples added 2 commits November 6, 2025 10:01

Storing changes commit

7ff1525

Working orchestrators + activities

552a2dd

andystaples requested a review from Copilot November 21, 2025 20:48

Copilot started reviewing on behalf of andystaples November 21, 2025 20:48 View session

Copilot finished reviewing on behalf of andystaples November 21, 2025 20:52

Copilot AI reviewed Nov 21, 2025

View reviewed changes

andystaples commented Nov 21, 2025

View reviewed changes

andystaples and others added 3 commits November 21, 2025 14:36

Nitpicks and cleanup

af0e3c2

Merge branch 'main' into andystaples/add-functions-support

86ee081

Save-all nits

3497148

andystaples added 5 commits November 24, 2025 11:59

Add entity support (needs extension change)

57de878

Refine entity support

18145f8

- Still needs eventSent and eventRecieved implementations

Finish entity support

9965ba4

Bump durabletask version, fix metadata

cc005ae

github-advanced-security bot found potential problems Dec 4, 2025

View reviewed changes

@@ -1,5 +1,8 @@
             name: Durable Task Scheduler SDK (durabletask-azurefunctions)
+            permissions:
+              contents: read
             on:
               push:
                 branches:

		# TODO: Is there a better way to support retrieving the unwrapped user code?
		df_client_middleware.client_function = fb._function._func # type: ignore

Andystaples/add functions support #75

Are you sure you want to change the base?

Andystaples/add functions support #75

Uh oh!

Conversation

andystaples commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hallvictoria Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

berndverst commented Nov 24, 2025

Uh oh!

Check warning

Copilot Autofix

Check warning

Copilot Autofix

Check warning

Copilot Autofix

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

andystaples commented Nov 21, 2025 •

edited

Loading

hallvictoria Dec 1, 2025 •

edited

Loading