-
Notifications
You must be signed in to change notification settings - Fork 251
examples: add tool execution safety guard #2193
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
March-77
wants to merge
3
commits into
trpc-group:main
Choose a base branch
from
March-77:agent/tool-safety-guard
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| # Tool Execution Safety Guard | ||
|
|
||
| This example scans tool commands before execution and returns `allow`, `deny`, | ||
| or `ask`. Policy changes require no code changes: edit | ||
| `tool_safety_policy.json` to configure allowed and denied commands, protected | ||
| paths, network domains, time/output limits, and environment allowlists. | ||
|
|
||
| ```bash | ||
| cd examples/tool_safety_guard | ||
| go test ./... | ||
| go run . \ | ||
| --policy tool_safety_policy.json \ | ||
| --samples samples.json \ | ||
| --report tool_safety_report.json \ | ||
| --audit tool_safety_audit.jsonl | ||
| ``` | ||
|
|
||
| The twelve samples are scanned only; dangerous commands are never executed. | ||
| `Guard.Wrap` is the execution-boundary adapter: it scans first, writes one audit | ||
| event, rejects `deny`/`ask`, and invokes the wrapped executor only for `allow`. | ||
| It can back an equivalent `tool.PermissionPolicy` for `workspaceexec`, | ||
| `hostexec`, or `codeexec`. | ||
|
|
||
| ## Security layers | ||
|
|
||
| - `internal/shellsafe` and this guard conservatively inspect command structure. | ||
| Unparsed wrappers, expansion, pipes, redirects, or unknown commands never | ||
| default to allow. | ||
| - Permission/Filter policy decides whether an invocation may proceed. It is the | ||
| pre-execution policy layer, not a runtime containment boundary. | ||
| - `workspaceexec` should use an isolated workspace, clean environment, output | ||
| bounds, and process cleanup. A workspace still needs a container/E2B sandbox | ||
| when executing untrusted code. | ||
| - `hostexec` runs on the host. PTY sessions, background processes, elevation, | ||
| and process-tree cleanup carry higher risk and require review. | ||
| - `codeexecutor/container` or E2B supplies network/filesystem/process isolation. | ||
| Static scanning cannot see every runtime behavior, while a sandbox cannot | ||
| determine business authorization; both layers are required. | ||
| - Audit JSONL is suitable for monitoring. The result exposes | ||
| `tool.safety.decision`, `tool.safety.risk_level`, `tool.safety.rule_id`, and | ||
| `tool.safety.backend` as OpenTelemetry-ready span attributes. | ||
|
|
||
| Secrets in command evidence are redacted before reports. Production callers | ||
| must also redact executor output, logs, artifacts, span events, and errors after | ||
| execution. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| module trpc.group/trpc-go/trpc-agent-go/examples/tool_safety_guard | ||
|
|
||
| go 1.23.0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,272 @@ | ||
| // Tencent is pleased to support the open source community by making trpc-agent-go available. | ||
| // Copyright (C) 2025 Tencent. All rights reserved. | ||
| // trpc-agent-go is licensed under the Apache License Version 2.0. | ||
| package main | ||
|
|
||
| import ( | ||
| "context" | ||
| "encoding/json" | ||
| "fmt" | ||
| "net/url" | ||
| "os" | ||
| "path/filepath" | ||
| "regexp" | ||
| "strconv" | ||
| "strings" | ||
| "time" | ||
| ) | ||
|
|
||
| type Policy struct { | ||
| AllowedCommands []string `json:"allowed_commands"` | ||
| DeniedCommands []string `json:"denied_commands"` | ||
| ForbiddenPaths []string `json:"forbidden_paths"` | ||
| AllowedDomains []string `json:"allowed_domains"` | ||
| MaxTimeoutSeconds int `json:"max_timeout_seconds"` | ||
| MaxOutputBytes int `json:"max_output_bytes"` | ||
| AllowedEnvVars []string `json:"allowed_env_vars"` | ||
| } | ||
| type Request struct { | ||
| ToolName string `json:"tool_name"` | ||
| Command string `json:"command"` | ||
| Backend string `json:"backend"` | ||
| WorkingDir string `json:"working_dir,omitempty"` | ||
| Environment map[string]string `json:"environment,omitempty"` | ||
| TimeoutSeconds int `json:"timeout_seconds,omitempty"` | ||
| MaxOutputBytes int `json:"max_output_bytes,omitempty"` | ||
| PTY bool `json:"pty,omitempty"` | ||
| } | ||
| type Finding struct { | ||
| Decision string `json:"decision"` | ||
| RiskLevel string `json:"risk_level"` | ||
| RuleID string `json:"rule_id"` | ||
| Evidence string `json:"evidence"` | ||
| Recommendation string `json:"recommendation"` | ||
| } | ||
| type ScanResult struct { | ||
| ToolName string `json:"tool_name"` | ||
| Command string `json:"command"` | ||
| Backend string `json:"backend"` | ||
| Decision string `json:"decision"` | ||
| RiskLevel string `json:"risk_level"` | ||
| RuleID string `json:"rule_id"` | ||
| Evidence string `json:"evidence"` | ||
| Recommendation string `json:"recommendation"` | ||
| Blocked bool `json:"blocked"` | ||
| Redacted bool `json:"redacted"` | ||
| DurationMicros int64 `json:"duration_micros"` | ||
| Findings []Finding `json:"findings"` | ||
| SpanAttributes map[string]string `json:"span_attributes"` | ||
| } | ||
| type AuditEvent struct { | ||
| Timestamp string `json:"timestamp"` | ||
| ToolName string `json:"tool_name"` | ||
| Decision string `json:"decision"` | ||
| RiskLevel string `json:"risk_level"` | ||
| RuleID string `json:"rule_id"` | ||
| Backend string `json:"backend"` | ||
| DurationMicros int64 `json:"duration_micros"` | ||
| Redacted bool `json:"redacted"` | ||
| Blocked bool `json:"blocked"` | ||
| } | ||
|
|
||
| func LoadPolicy(path string) (Policy, error) { | ||
| var p Policy | ||
| data, e := os.ReadFile(path) | ||
| if e != nil { | ||
| return p, e | ||
| } | ||
| e = json.Unmarshal(data, &p) | ||
| return p, e | ||
| } | ||
|
|
||
| type deniedCommandPattern struct { | ||
| command string | ||
| pattern *regexp.Regexp | ||
| } | ||
|
|
||
| type Guard struct { | ||
| policy Policy | ||
| deniedPatterns []deniedCommandPattern | ||
| } | ||
|
|
||
| func NewGuard(p Policy) *Guard { | ||
| g := &Guard{policy: p} | ||
| for _, command := range p.DeniedCommands { | ||
| command = strings.ToLower(command) | ||
| g.deniedPatterns = append(g.deniedPatterns, deniedCommandPattern{ | ||
| command: command, | ||
| pattern: regexp.MustCompile(`(^|[;&|\n]\s*)` + regexp.QuoteMeta(command) + `(\s|$)`), | ||
| }) | ||
| } | ||
| return g | ||
| } | ||
|
|
||
| var ( | ||
| secretRE = regexp.MustCompile(`(?i)(api[_-]?key|token|password|secret)\s*[=:]\s*[^\s]+`) | ||
| destructiveCommandRE = regexp.MustCompile(`\brm\s+[^\n]*(?:-rf|-fr)|\b(?:mkfs|dd)\b`) | ||
| secretReadRE = regexp.MustCompile(`(?i)(cat|type|grep|less|head|tail)\s+[^\n]*(\.env|credentials|id_rsa|\.ssh)`) | ||
| networkCommandRE = regexp.MustCompile(`\b(curl|wget|nc|netcat|ssh)\b`) | ||
| shellWrapperRE = regexp.MustCompile(`\b(?:sh|bash|cmd|powershell)\s+(?:-c|/c|-command)\b|\beval\b`) | ||
| dependencyChangeRE = regexp.MustCompile(`\b(go\s+install|npm\s+install|pip\s+install|apt(?:-get)?\s+install)\b`) | ||
| unboundedExecutionRE = regexp.MustCompile(`\b(while\s+true|for\s*\(\s*;\s*;|yes\b|fork\s*bomb)`) | ||
| sleepRE = regexp.MustCompile(`\bsleep\s+(\d+)`) | ||
| ) | ||
|
|
||
| func redact(s string) (string, bool) { | ||
| out := secretRE.ReplaceAllStringFunc(s, func(v string) string { | ||
| if i := strings.IndexAny(v, "=:"); i >= 0 { | ||
| return v[:i+1] + "***REDACTED***" | ||
| } | ||
| return "***REDACTED***" | ||
| }) | ||
| return out, out != s | ||
| } | ||
| func (g *Guard) Scan(req Request) ScanResult { | ||
| started := time.Now() | ||
| command, redacted := redact(strings.TrimSpace(req.Command)) | ||
| r := ScanResult{ToolName: req.ToolName, Command: command, Backend: req.Backend, Decision: "allow", RiskLevel: "low", RuleID: "ALLOW_POLICY", Evidence: "command satisfies configured policy", Recommendation: "execute only in the configured sandbox", Redacted: redacted} | ||
| add := func(decision, risk, id, evidence, recommendation string) { | ||
| ev, changed := redact(evidence) | ||
| r.Redacted = r.Redacted || changed | ||
| r.Findings = append(r.Findings, Finding{Decision: decision, RiskLevel: risk, RuleID: id, Evidence: ev, Recommendation: recommendation}) | ||
| } | ||
| lower := strings.ToLower(req.Command) | ||
| tokens := strings.Fields(lower) | ||
| base := "" | ||
| if len(tokens) > 0 { | ||
| base = strings.Trim(tokens[0], "'\"") | ||
| } | ||
| for _, denied := range g.deniedPatterns { | ||
| if base == denied.command || denied.pattern.MatchString(lower) { | ||
| add("deny", "critical", "DENIED_COMMAND", denied.command, "remove the denied command") | ||
| } | ||
| } | ||
| if destructiveCommandRE.MatchString(lower) { | ||
| add("deny", "critical", "DESTRUCTIVE_COMMAND", req.Command, "do not run destructive filesystem commands") | ||
| } | ||
| workingDir := strings.ToLower(filepath.ToSlash(filepath.Clean(req.WorkingDir))) | ||
| for _, path := range g.policy.ForbiddenPaths { | ||
| normalizedPath := strings.ToLower(filepath.ToSlash(filepath.Clean(path))) | ||
| if strings.Contains(lower, strings.ToLower(path)) { | ||
| add("deny", "critical", "FORBIDDEN_PATH", path, "remove access to the protected path") | ||
| } | ||
| if req.WorkingDir != "" && strings.Contains(workingDir, normalizedPath) { | ||
| add("deny", "critical", "FORBIDDEN_WORKING_DIR", req.WorkingDir, "choose a working directory outside protected paths") | ||
| } | ||
| } | ||
| if secretReadRE.MatchString(req.Command) { | ||
| add("deny", "critical", "SECRET_READ", req.Command, "use a secret provider instead of reading credential files") | ||
| } | ||
| if networkCommandRE.MatchString(lower) { | ||
| hosts := extractHosts(req.Command) | ||
| if len(hosts) == 0 { | ||
| add("ask", "high", "NETWORK_UNPARSED", req.Command, "provide a literal allowlisted HTTPS URL") | ||
| } | ||
| for _, host := range hosts { | ||
| if !containsFold(g.policy.AllowedDomains, host) { | ||
| add("deny", "critical", "NETWORK_NOT_ALLOWLISTED", host, "add a reviewed domain to allowed_domains or remove the request") | ||
| } | ||
| } | ||
| } | ||
| if shellWrapperRE.MatchString(lower) { | ||
| add("ask", "high", "SHELL_WRAPPER", req.Command, "expand and review the wrapped command before execution") | ||
| } | ||
| if strings.ContainsAny(req.Command, "`$") || strings.Contains(req.Command, "|") || strings.ContainsAny(req.Command, ";><") { | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Newline separators bypass denied command checks, so 中文换行分隔符会绕过禁用命令检查,因此 `go test ./...` 后接 `sudo` 会返回 `allow`。请在返回 `allow` 前将换行视为命令分隔符。 |
||
| add("ask", "high", "SHELL_METACHAR", req.Command, "use argv execution without shell metacharacters") | ||
| } | ||
| if dependencyChangeRE.MatchString(lower) { | ||
| add("ask", "high", "DEPENDENCY_CHANGE", req.Command, "pin and review dependencies in an isolated build environment") | ||
| } | ||
| if unboundedExecutionRE.MatchString(lower) { | ||
| add("deny", "critical", "UNBOUNDED_EXECUTION", req.Command, "replace unbounded work with a bounded operation") | ||
| } | ||
| if m := sleepRE.FindStringSubmatch(lower); len(m) > 1 { | ||
| n, _ := strconv.Atoi(m[1]) | ||
| if n > g.policy.MaxTimeoutSeconds { | ||
| add("ask", "medium", "LONG_RUNNING", m[0], "reduce duration below the configured timeout") | ||
| } | ||
| } | ||
| if req.TimeoutSeconds > g.policy.MaxTimeoutSeconds { | ||
| add("deny", "high", "TIMEOUT_LIMIT", fmt.Sprint(req.TimeoutSeconds), "use a timeout within policy") | ||
| } | ||
| if req.MaxOutputBytes > g.policy.MaxOutputBytes { | ||
| add("deny", "high", "OUTPUT_LIMIT", fmt.Sprint(req.MaxOutputBytes), "lower the maximum output size") | ||
| } | ||
| if req.Backend == "hostexec" && (req.PTY || strings.Contains(lower, "&")) { | ||
| add("ask", "high", "HOST_SESSION", req.Command, "prefer workspaceexec; require approval and process-tree cleanup") | ||
| } | ||
| for key := range req.Environment { | ||
| if !containsFold(g.policy.AllowedEnvVars, key) { | ||
| add("ask", "medium", "ENV_NOT_ALLOWLISTED", key, "remove the environment variable or add it after review") | ||
| } | ||
| } | ||
| if len(tokens) == 0 { | ||
| add("deny", "high", "EMPTY_COMMAND", "empty command", "provide an explicit argv command") | ||
| } else if !containsFold(g.policy.AllowedCommands, base) && len(r.Findings) == 0 { | ||
| add("ask", "medium", "COMMAND_NOT_ALLOWLISTED", base, "add the command to allowed_commands after review") | ||
| } | ||
| for _, f := range r.Findings { | ||
| if rank(f.Decision) > rank(r.Decision) { | ||
| r.Decision, r.RiskLevel, r.RuleID, r.Evidence, r.Recommendation = f.Decision, f.RiskLevel, f.RuleID, f.Evidence, f.Recommendation | ||
| } | ||
| } | ||
| r.Blocked = r.Decision != "allow" | ||
| r.DurationMicros = time.Since(started).Microseconds() | ||
| r.SpanAttributes = map[string]string{"tool.safety.decision": r.Decision, "tool.safety.risk_level": r.RiskLevel, "tool.safety.rule_id": r.RuleID, "tool.safety.backend": r.Backend} | ||
| return r | ||
| } | ||
| func rank(v string) int { | ||
| switch v { | ||
| case "deny": | ||
| return 3 | ||
| case "ask", "needs_human_review": | ||
| return 2 | ||
| case "allow": | ||
| return 1 | ||
| } | ||
| return 0 | ||
| } | ||
| func containsFold(values []string, want string) bool { | ||
| for _, v := range values { | ||
| if strings.EqualFold(v, want) { | ||
| return true | ||
| } | ||
| } | ||
| return false | ||
| } | ||
| func extractHosts(s string) []string { | ||
| var out []string | ||
| for _, word := range strings.Fields(s) { | ||
| word = strings.Trim(word, "'\";,|<>") | ||
| if u, e := url.Parse(word); e == nil && u.Hostname() != "" { | ||
| out = append(out, u.Hostname()) | ||
| } | ||
| } | ||
| return out | ||
| } | ||
|
|
||
| type Executor func(context.Context, Request) (string, error) | ||
|
|
||
| func (g *Guard) Wrap(next Executor, audit func(AuditEvent) error) Executor { | ||
| return func(ctx context.Context, req Request) (string, error) { | ||
| result := g.Scan(req) | ||
| event := AuditEvent{ | ||
| Timestamp: time.Now().UTC().Format(time.RFC3339Nano), | ||
| ToolName: req.ToolName, | ||
| Decision: result.Decision, | ||
| RiskLevel: result.RiskLevel, | ||
| RuleID: result.RuleID, | ||
| Backend: req.Backend, | ||
| DurationMicros: result.DurationMicros, | ||
| Redacted: result.Redacted, | ||
| Blocked: result.Blocked, | ||
| } | ||
| if e := audit(event); e != nil { | ||
| return "", e | ||
| } | ||
| if result.Blocked { | ||
| return "", fmt.Errorf("tool execution blocked: %s (%s)", result.RuleID, result.Decision) | ||
| } | ||
| return next(ctx, req) | ||
| } | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ForbiddenPathsonly checksreq.Command, soWorkingDir: "~/.ssh"with a harmless command can be allowed even though the guard says it protects working directories. Normalize and checkreq.WorkingDiragainstForbiddenPathsas well, then add a sample/test for protected working dirs.中文
`ForbiddenPaths` 现在只检查 `req.Command`,所以即使 `WorkingDir` 是 `~/.ssh`,只要命令本身看起来无害就可能被放行,这和 guard 宣称会保护工作目录不一致。建议把 `req.WorkingDir` 规范化后也纳入 `ForbiddenPaths` 检查,并补一个受保护工作目录的样例/测试。