test(agent-mesh): make approval timeout test deterministic - #3839
Open
Buriro Chimodoi Ezekia (buriro-ezekia) wants to merge 2 commits into
Open
Conversation
Signed-off-by: Buriro Chimodoi Ezekia <93260631+buriro-ezekia@users.noreply.github.com>
Signed-off-by: Buriro Chimodoi Ezekia <93260631+buriro-ezekia@users.noreply.github.com>
Buriro Chimodoi Ezekia (buriro-ezekia)
requested review from
MohammadHaroonAbuomar and
liamcrumm
as code owners
August 26, 2026 18:17
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
|
Welcome to the Agent Governance Toolkit! Thanks for your first pull request. |
PR Review Summary
Verdict: AI review comments are untrusted advisory output. The summary reports workflow-generated completion status only, not model-authored pass/fail claims. |
Author
|
@microsoft-github-policy-service agree |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issue
No dedicated issue. The affected regression was introduced as part of #3106.
Problem & Solution
Problem:
TestCallbackTimeoutDeprecation.test_timeout_always_deniesrelied ontime.sleep(0.005)and a subsequenttime.monotonic()delta. On Windows, the monotonic clock in the local test environment reported a resolution of 15.625 ms, so a 5 ms sleep could still produce an observed elapsed time of0.0. Withtimeout_seconds=0, the test could therefore return the callback's approval instead of exercising the timeout-denial branch.This made the security regression test intermittently fail even though the implementation remained fail-closed when an elapsed timeout was actually observed.
Solution: replace the real-clock sleep with deterministic monotonic values in the test. The test patches the exact
CallbackApproval.request_approvalfunction-global clock for the duration of the test, so it always observes a positive elapsed interval and verifies the intendedsystem:timeoutdenial without depending on host timer granularity.Production approval code is unchanged.
Impact on Your Work
The flaky test blocked a clean AgentMesh approval-test baseline on Windows and made repeated local verification unreliable. This change preserves the existing security assertion while making the regression deterministic and portable across platforms.
Timeline
None.
Alternatives Considered
time.monotonicthrough a separately imported module alias. This was tested first, but the consolidated package layout can load the implementation through a different module object in editable or local installations. Patching the function's own globals targets the clock actually resolved by the implementation.Type of Change
Package(s) Affected
Core & runtime:
Governance & security:
Platform & tooling:
CLI plugins:
Shared / other:
Testing
Unit Testing
76 passed.Manual Testing
Verified locally on Windows with Python 3.12:
python -m pip check—No broken requirements found.python -m ruff check agent-governance-python/agent-mesh/tests/test_approval_strict.py—All checks passed!Checklist
The full AgentMesh package suite has not been run locally; the focused approval suite passed cleanly. The CLA checkbox is intentionally left open for the repository's CLA bot to confirm.
Attribution & Prior Art
Prior art / related projects (if any):
None.
AI Assistance
ChatGPT was used to help diagnose the Windows timer-resolution failure and draft the deterministic test change. The exact diff was reviewed and the validation commands were run locally before submission.
IP, Patents, and Licensing