Add deferrable mode to AzureContainerInstancesOperator

[cruseakshay]

Add deferrable=True support to AzureContainerInstancesOperator so
tasks release their worker slot while waiting for long-running containers,
offloading polling to the lightweight Triggerer process.

Changes

• New AzureContainerInstanceAsyncHook — async counterpart to the
  existing sync hook, using azure.mgmt.containerinstance.aio for
  non-blocking state/log/delete calls.

• New AzureContainerInstanceTrigger — polls ACI at a configurable
  interval and yields a TriggerEvent when the container reaches a
  terminal state.

• AzureContainerInstancesOperator — three new parameters:

  • deferrable (default from conf) — enables deferrable mode
  • polling_interval (default 5.0s) — trigger poll frequency
  • remove_on_success (default True) — controls cleanup on success
  • Also fixes a bug where the finally cleanup block would delete a
    still-running container group immediately after self.defer() raised
    TaskDeferred.

• provider.yaml + get_provider_info.py updated to register the
  new trigger.

closes: #62433

---

Was generative AI tooling used to co-author this PR?

• Yes — Cursor, following the guidelines

[SameerMesiah97]

The overall approach is solid, though the style is a bit different from the deferrable mode implementation for the ADF counterpart. I have left a few comments for you to address. On a slightly tangential point, the PR is quite large so, for future reference, I would advise splitting a PR like this into the following:

• PR 1: Async Hook
• PR 2: Deferrable mode implementation (Operator + Trigger)
• PR 3: Bug fix

Combining all 3 in one PR makes it harder to review. You need not do this now but please keep this in mind the next time you submit a PR.

[potiuk]

@cruseakshay This PR has been converted to draft because it does not yet meet our Pull Request quality criteria.

Issues found:

• ❌ Merge conflicts: This PR has merge conflicts with the main branch. Your branch is 340 commits behind main. Please rebase your branch (git fetch origin && git rebase origin/main), resolve the conflicts, and push again. See contributing quick start.

Note: Your branch is 340 commits behind main. Some check failures may be caused by changes in the base branch rather than by your PR. Please rebase your branch and push again to get up-to-date CI results.

What to do next:

• The comment informs you what you need to do.
• Fix each issue, then mark the PR as "Ready for review" in the GitHub UI - but only after making sure that all the issues are fixed.
• Maintainers will then proceed with a normal review.

Converting a PR to draft is not a rejection — it is an invitation to bring the PR up to the project's standards so that maintainer review time is spent productively. If you have questions, feel free to ask on the Airflow Slack.

[dabla]

Don't really like the contextmanager on hook level, I would prefer to see this on the get_async_conn method instead, as the the connection we want to make sure is closed correctly in case of exception, not the hook, as the hook is just a way to get the connection, but it isn't the resource itself, the connection is.

So more something like this:

@asynccontextmanager
async def get_async_conn(self):
    credential = ...
    client = AsyncContainerInstanceManagementClient(...)
    try:
        yield client
    finally:
        await client.close()
        if hasattr(credential, "close"):
            await credential.close()

So later on it can be used like this in the trigger:

async with hook.get_async_conn() as client:
    await client.container_groups.get(...)

But I really like the fact that you implemented a contextmanager, because that's a clean way to cope with connections.

[cruseakshay]

Thanks @dabla! I will work on asynccontextmanager as you mentioned.

[cruseakshay]

@dabla I have added asynccontextmanager, PTAL.