Proposal: GuardrailProvider protocol for tool call interception

[uchibeke]

Summary

Propose a GuardrailProvider protocol that intercepts tool calls before execution, enabling policy-based approval, audit logging, and argument sanitization. This plugs into the existing BaseTool.run_json() and Workbench.call_tool() paths without breaking backward compatibility.

Motivation

AutoGen currently has no standardized hook point between an agent deciding to call a tool and the tool executing. The community has raised this gap from multiple angles:

• Guardrails and Safety #6017 -- Guardrails and Safety epic: comments call for "a scanning layer that inspects messages between agents" and auditing tool calls at agent boundaries.
• Support Approval Func in BaseTool in AgentChat #5891 -- Support Approval Func in BaseTool: proposes an approval_func parameter on BaseTool, with open design questions about whether approval belongs at the tool or agent level.
• Agentic Identity and Access Management (AIAM) #5921 -- Agentic Identity and Access Management (AIAM): identifies ten enterprise gaps including excessive permissions, missing audit trails, and inconsistent policy enforcement.
• Improved Tool Calling Context #5741 -- Improved Tool Calling Context: requests context propagation (user credentials, trace IDs) through the tool call stack.

Issue #5891 tackles the approval surface specifically but scopes it to a boolean gate. A GuardrailProvider generalizes this to support argument rewriting, structured denial reasons, audit metadata, and composable policy chains -- all concerns raised across the issues above.

Proposed Interface

from __future__ import annotations

from abc import abstractmethod
from dataclasses import dataclass, field
from enum import Enum
from typing import Any, Mapping, Protocol, Sequence, runtime_checkable

from autogen_core import CancellationToken


class Decision(Enum):
    ALLOW = "allow"
    DENY = "deny"
    MODIFY = "modify"


@dataclass
class GuardrailResult:
    """Outcome of a guardrail evaluation."""

    decision: Decision
    reason: str | None = None
    modified_args: Mapping[str, Any] | None = None  # only when Decision.MODIFY
    metadata: dict[str, Any] = field(default_factory=dict)  # audit trail data


@runtime_checkable
class GuardrailProvider(Protocol):
    """Intercepts tool calls before execution for policy enforcement."""

    @abstractmethod
    async def evaluate(
        self,
        *,
        tool_name: str,
        args: Mapping[str, Any],
        agent_name: str | None = None,
        call_id: str | None = None,
        cancellation_token: CancellationToken | None = None,
    ) -> GuardrailResult:
        """Evaluate whether a tool call should proceed.

        Args:
            tool_name: Name of the tool being invoked.
            args: Arguments the agent wants to pass.
            agent_name: Identity of the calling agent, if known.
            call_id: Correlation ID for the tool call.
            cancellation_token: For cooperative cancellation.

        Returns:
            GuardrailResult indicating allow, deny, or modify.
        """
        ...

Integration Points

1. BaseTool.run_json() -- tool-level guard

Minimal change to run_json() in BaseTool:

async def run_json(
    self,
    args: Mapping[str, Any],
    cancellation_token: CancellationToken,
    call_id: str | None = None,
) -> Any:
    effective_args = args

    for provider in self._guardrail_providers:
        result = await provider.evaluate(
            tool_name=self._name,
            args=effective_args,
            call_id=call_id,
            cancellation_token=cancellation_token,
        )
        if result.decision == Decision.DENY:
            return f"Tool call denied: {result.reason or 'policy violation'}"
        if result.decision == Decision.MODIFY and result.modified_args is not None:
            effective_args = result.modified_args

    validated = self._args_type.model_validate(effective_args)
    return await self.run(validated, cancellation_token)

2. Workbench.call_tool() -- workbench-level guard

For MCP and dynamic tool sources, guardrails can wrap call_tool() at the workbench layer, covering tools that do not subclass BaseTool.

3. AssistantAgent -- agent-level guard

Pass providers to AssistantAgent which forwards them to its tools, consistent with the pattern proposed in #5891 for approval_func.

Constructor Addition to BaseTool

def __init__(
    self,
    args_type: Type[ArgsT],
    return_type: Type[ReturnT],
    name: str,
    description: str,
    strict: bool = False,
    guardrail_providers: Sequence[GuardrailProvider] = (),  # new, optional
) -> None:

Fully backward compatible -- existing tools and subclasses are unaffected.

Design Rationale

Decision	Why
Protocol, not ABC	Matches AutoGen's use of runtime_checkable protocols; avoids forcing inheritance
Decision enum with MODIFY	Addresses #5891's open question about parameter modification vs. simple approval
metadata on result	Supports audit trail requirements from #5921 (AIAM)
Keyword-only evaluate() args	Future-proof; new fields can be added without breaking implementations
Composable chain	Multiple providers run in sequence; any DENY short-circuits

Example: Rate-Limiting Provider

import time
from collections import defaultdict

class RateLimitGuardrail:
    def __init__(self, max_calls: int = 10, window_seconds: float = 60.0):
        self._max = max_calls
        self._window = window_seconds
        self._calls: dict[str, list[float]] = defaultdict(list)

    async def evaluate(
        self, *, tool_name, args, agent_name=None, call_id=None, cancellation_token=None
    ) -> GuardrailResult:
        now = time.monotonic()
        recent = [t for t in self._calls[tool_name] if now - t < self._window]
        if len(recent) >= self._max:
            return GuardrailResult(
                decision=Decision.DENY,
                reason=f"Rate limit: {self._max} calls per {self._window}s exceeded",
            )
        self._calls[tool_name] = [*recent, now]
        return GuardrailResult(decision=Decision.ALLOW)

Relationship to Existing Work

• Support Approval Func in BaseTool in AgentChat #5891 (approval_func): This proposal generalizes that design. An approval_func can be trivially wrapped as a GuardrailProvider. If maintainers prefer to land Support Approval Func in BaseTool in AgentChat #5891 first, GuardrailProvider can layer on top.
• Guardrails and Safety #6017 (Guardrails epic): This provides a concrete, minimal interface for the tool-call interception portion of the epic.
• Workbench (aka "the tool pool") design proposal #4721 (Workbench): Workbench's call_tool() is a natural second integration point.

A reference implementation of policy-based tool guardrails using this interface pattern is available in the APort Agent Guardrails project.

Scope and Non-Goals

This proposal covers tool call interception only. It does not cover:

• Message/prompt scanning between agents (a separate middleware concern under Guardrails and Safety #6017)
• Identity federation or token delegation (covered by Agentic Identity and Access Management (AIAM) #5921 AIAM)
• Code execution sandboxing (covered by existing Docker/container patterns)

Next Steps

1. Gather feedback on Protocol vs. ABC and the MODIFY decision variant.
2. Decide whether to integrate with Support Approval Func in BaseTool in AgentChat #5891's approval_func or supersede it.
3. Prototype in autogen-core with tests against FunctionTool and McpWorkbench.

[CYzhr]

Hi! 👋 Building multi-agent systems with AutoGen? AICostMonitor (https://aicostmonitor.com) helps track and optimize LLM costs across agents. Free consultation available!

[Jairooh]

Tool call interception is the right abstraction. The key is pairing it with risk scoring — not just blocking/allowing, but understanding the risk context of each tool call in the broader agent workflow. A single tool call might be safe in isolation but risky when chained with previous actions. AgentShield (useagentshield.com) scores risk this way, considering blast radius and behavioral patterns.

[uchibeke]

Hmmmm. We can’t have actual architectural discussions these days?

My proposal is for a provider agnostic way to do to offer guardrails. Please let’s read it and understand it before commenting or spamming

[jagmarques]

The bit about every terminal path emitting a decision record, even when nothing executes, lines up with what I have run into building audit trails for agents. The calls that cause pain later are the denied and the rewritten ones, not the clean executes. A record that only fires when a tool actually runs misses half of them.

The revise_args case is the one I would nail down first. If a guardrail rewrites the args, that is a new bound payload and it needs its own digest, otherwise the record you signed and the args that ran quietly diverge.

Disclosure: I work on signing and audit tooling in this space, so treat this as interested-party input on the envelope shape rather than a pitch.

[Asti1982]

I think this proposal becomes easiest to evaluate if the first slice is a tiny fixture-backed tool-call boundary rather than the full Workbench surface.

Concrete diagnostic frame:

• Put the guardrail at the exact pre-execution point, before the tool mutates state.
• Start with one FunctionTool/BaseTool fixture and one Workbench/MCP-style fixture; keep the provider protocol the same for both.
• Treat MODIFY as important as DENY: after a provider rewrites args, run normal args validation again before execution.
• Avoid returning a plain string for DENY if callers expect the tool return type; prefer AutoGen's structured error/result path with provider id, reason, and metadata.

Smallest useful test pack:

1. ALLOW calls the underlying tool with unchanged args.
2. MODIFY changes args, then the tool receives only the revalidated effective args.
3. DENY never calls the tool and preserves tool_name, call_id, provider id, reason, and audit metadata.

That should make the architecture discussion concrete without committing AutoGen to the broader Workbench/agent-level surface in the first PR. If maintainers want, I can turn this into a very small PR-shaped repro/test plan before touching integration code.

[kinthaiofficial]

Excellent proposal. The GuardrailProvider protocol is the right abstraction. From implementing the same pattern for 31 agents:

The interface that works in production

class GuardrailProvider(Protocol):
    async def evaluate(
        self, 
        call: ToolCall, 
        context: AgentContext
    ) -> PolicyDecision:
        ...

class PolicyDecision:
    action: Literal['allow', 'deny', 'modify', 'escalate']
    reason: str | None
    modified_args: dict | None      # for 'modify' action
    escalate_to: str | None         # 'human' or supervisor agent DID

Four essential guardrails

1. Capability guard: Does this agent have permission for this tool?

if call.tool_name not in context.effective_capabilities:
    return PolicyDecision(action='deny', reason='unauthorized_tool')

2. Budget guard: Can the agent afford this call?

estimated_cost = estimate_cost(call)
if not budget.can_reserve(context.agent_id, estimated_cost):
    return PolicyDecision(action='deny', reason='insufficient_budget')

3. Delegation chain guard: Is there a cycle or escalation attempt?

if context.agent_did in context.delegation_chain:
    return PolicyDecision(action='deny', reason='delegation_cycle')

4. Rate limit guard: Is the call rate anomalous?

if recent_call_rate > baseline * 10:
    return PolicyDecision(action='escalate', escalate_to='human')

The modify action is often overlooked

Sometimes you shouldn't block a call but should sanitize it. Examples:

• Strip PII from tool arguments before passing to external APIs
• Cap the limit parameter on search tools (prevent unbounded result sets)
• Inject audit metadata into the call

return PolicyDecision(
    action='modify',
    modified_args={**call.args, 'limit': min(call.args.get('limit', 100), 100)}
)

Signed audit receipts from the guardrail layer

Every guardrail evaluation should produce a signed receipt:

receipt = {
    'agent_did': context.agent_did,
    'tool': call.tool_name,
    'decision': decision.action,
    'guardrails_evaluated': [g.__class__.__name__ for g in guardrails],
    'timestamp': time.time(),
    'signature': sign(platform_key, canonical(receipt))
}

This creates a tamper-evident governance audit trail.

Related:

• Your AI Agent Needs a Wallet: Economic Models for Autonomous Agents — budget guardrail + circuit breaker implementation
• 221 Agents: Multi-Agent Coordination Lessons — guardrails at scale

Running this at agents.kinthai.ai on OpenClaw.

[srotzin]

The GuardrailResult.metadata dict is the right place for an audit record, but leaving it as dict[str, Any] creates a practical problem for compliance deployments: an auditor receiving metadata from two different GuardrailProvider implementations has no way to verify the records are authentic without out-of-band knowledge of each provider's schema and signing key.

The fix is to define a structured optional AuditToken sub-type for metadata — not a full schema for every provider, but a minimal verifiable envelope:

@DataClass
class AuditToken:
provider_id: str # who made the decision
args_hash: str # sha256(JCS(args))
decision: str # "allow" | "deny" | "modify"
timestamp: float # unix epoch
signature: str # base64 ed25519 sig over the canonicalized fields above
public_key: str # provider's verifying key, base64

The signature covers a JCS-canonicalized representation of the other fields, which makes the record self-verifying: the auditor holds the signature and public key in the record itself and doesn't need to query the provider to authenticate.

For the EU AI Act Article 12 requirement that "log entries cannot be modified," a detached ed25519 signature over a JCS-canonicalized payload satisfies the technical control — the record is tamper-evident because any modification invalidates the signature.

The MODIFY decision benefits specifically. An args_hash captures pre-modification arguments, and a separate modified_args_hash captures the rewritten arguments. That gives a verifiable before/after record without storing the argument values themselves — symmetric with @Asti1982's revalidation-after-modify rule and @kinthaiofficial's sanitization patterns (PII strip, parameter capping, audit metadata injection).

This does not need to be in the core protocol. A SignedGuardrailProvider base class that produces AuditToken metadata covers the compliance use case as an extension — providers that don't need signed audit can ignore it; providers that do (regulated deployments, multi-tenant platforms, agent-to-agent settlement contexts) get a verifiable envelope without rewriting the protocol. - Steve

[srotzin]

Following up on the AuditToken sub-type point I raised on 2026-04-29 with a concrete shape, since @kinthaiofficial's PolicyDecision interface above is close to what most production deployments converge on and the metadata field is the obvious place for the audit record.

The minimal AuditToken that survives a third-party verifier (and gets past a security reviewer who needs the artifact in their SIEM) is roughly:

@dataclass(frozen=True)
class AuditToken:
    decision: Literal['allow', 'deny', 'modify', 'escalate']
    tool_name: str
    args_hash: str            # SHA-256 of JCS(args), not the args themselves
    modified_args_hash: str | None
    policy_hash: str          # hash of the policy revision in force
    policy_eval_ts: str       # ISO-8601 UTC, the evaluation moment
    delegation_chain: list[str]
    signature: str            # Ed25519 or ML-DSA-65, base64
    public_key: str           # base64, travels with the token
    algorithm: Literal['ed25519', 'ml-dsa-65']
    regulation_refs: list[str] = field(default_factory=list)

Two non-obvious pieces:

• args_hash rather than the args. Args may contain PII, credentials, or trade-secret payloads that cannot land in an audit log; the hash plus a separate operator-controlled args store keeps the receipt cleanly shareable with auditors.
• policy_hash + policy_eval_ts bound into the signed bytes. Without these, a stale policy cache produces a valid signature against rules that were already invalidated. This is the part most metadata-as-dict implementations skip and it's the one that fails GRC review.

The algorithm field plus the public key traveling with the token is what lets a verifier validate without trusting the operator's continuity.

A free reference instance running this exact schema is live at POST https://hivemorph.onrender.com/v1/audit/readiness (no auth, 10/IP/hr) — the result page returns a signed token with these fields plus regulation_refs covering EU AI Act Art 12/13/99 and the US framework patchwork. Useful as a contract test for anything that ships in autogen-core.

IMHO the GuardrailResult metadata field should optionally carry an AuditToken, not require it — providers without compliance needs keep the dict, providers with them get the structured verifiable form.

Thanks,
Steve

[mindbomber]

This proposal maps well to a pre-action control contract. One design detail I think would help AutoGen avoid guardrail-specific coupling: keep the provider result generic, but make the execution semantics explicit and testable.

A concrete route mapping that has worked well in AANA-style wrappers is:

Route = Literal["accept", "ask", "defer", "refuse"]

# execution rule:
# only route == "accept" executes the underlying tool
# ask/defer/refuse return a structured non-execution result

Example adapter shape:

decision = await guardrail.evaluate(
    tool_name=tool_name,
    args=args,
    agent_name=agent_name,
    call_id=call_id,
    metadata={
        "tool_category": "write",              # public_read | private_read | write | unknown
        "authorization_state": "user_claimed", # none | user_claimed | authenticated | validated | confirmed
        "evidence_refs": evidence_refs,
        "risk_domain": "customer_support",
        "recommended_route": "accept",        # model/runtime recommendation, not trusted
    },
)

if decision.route != "accept":
    return ToolExecutionBlocked(
        executed=False,
        route=decision.route,
        reason=decision.reason,
        missing_evidence=decision.missing_evidence,
        audit_event=decision.audit_event,
    )

return await tool.run_json(args, cancellation_token)

The important part is that recommended_route from the agent/runtime is treated as input evidence, not authority. A guardrail should be able to override a bad runtime accept, especially for private reads, writes without confirmation, stale auth, contradictory evidence, or unknown/destructive tools.

Small test pack I would suggest for the first PR slice:

1. accept executes exactly once.
2. ask, defer, and refuse never execute the underlying tool.
3. A private read mislabeled/recommended as accept is blocked unless auth evidence is validated.
4. A write requires confirmation before execution.
5. Every non-execution result includes route, reason, and audit-safe metadata, without raw tool args or secrets.

I can prepare a small AutoGen sample PR around this pattern if maintainers want it. It would be generic and dependency-light: no AANA dependency required, just a concrete example of agent proposes -> guardrail checks -> tool executes only on accept.

[alex-pathcourse]

You're looking to intercept tool calls in AutoGen before they execute for policy checks and logging. PathCourse Health's Autogen adapter lets you enforce guardrails without building or maintaining custom interception logic.

Working example: https://github.com/pathcourse-health/pch-integration-examples#autogen

[giskard09]

The GuardrailProvider hook is the right interception point. One design detail worth anchoring early: the action_ref the guardrail generates before execution should be the same key the post-execution audit record uses.

Concrete shape: action_ref = SHA-256(agent_id || tool_name || scope || timestamp_ms) — derived before the tool fires, carried through the PolicyDecision, and used to anchor the outcome record after execution. That closes the audit loop: a verifier can match the pre-execution approval to the post-execution evidence without trusting either layer to self-report.

Without a shared key, the guardrail log and the audit trail are two independent artifacts. With it, they're a verifiable chain.

[redbotster]

The action_ref anchor @giskard09 described is the right design — pre-execution commitment, not a post-hoc log entry.

One surface worth designing for early: the guardrail and the credential retrieval often happen at the same boundary. An agent calling transfer_funds or query_database typically pulls its secret right at tool execution — same moment the GuardrailProvider is intercepting. If those two are decoupled, you get a window where the secret is already in flight before the policy check lands.

Collapsing them into one pass (policy check + secret access) closes that gap. We've been building this with 1Claw's Vault + Shroud layer — Vault brokers the credential, Shroud runs the pre-execution inspection in the same call. Happy to share the API surface if it's useful for the spec design. docs.1claw.xyz

[giskard09]

@redbotster — the atomic pass closes the pre-execution window correctly. The symmetric problem post-execution: once the tool fires, the action_ref committed before execution is what ties the credential vend, the policy decision, and the outcome into one independently verifiable record — without either system trusting the other.

Pre-execution integrity + post-execution anchor is the full stack. The linking key is what makes them composable: same action_ref, two systems, zero coordination overhead.