From PR #346 review (bfcl_v4_execution.py:235).
Decoded tool-call strings (function name + arguments straight from the endpoint's tool_calls response) reach bfcl-eval's eval(func_call), guarded only by an upstream name denylist (kill/exit/quit/remove/unlink/popen/Popen/run). A tool call named __import__('os').system bypasses it (split("(")[0] yields __import__, not on the list), so a compromised or untrusted --endpoint gets arbitrary code execution on the evaluator host.
This is upstream bfcl-eval behavior, but this PR is what wires arbitrary-endpoint HTTP responses into it.
Options:
- At minimum: document that multi-turn eval must only target trusted endpoints.
- Better: validate
tc["name"] against the entry's known tool names in process_response before handing calls to bfcl-eval.
Deferred out of PR #346 to keep it focused; the multi-turn live-execution path is opt-in (its own CLI).
From PR #346 review (
bfcl_v4_execution.py:235).Decoded tool-call strings (function name + arguments straight from the endpoint's
tool_callsresponse) reach bfcl-eval'seval(func_call), guarded only by an upstream name denylist (kill/exit/quit/remove/unlink/popen/Popen/run). A tool call named__import__('os').systembypasses it (split("(")[0]yields__import__, not on the list), so a compromised or untrusted--endpointgets arbitrary code execution on the evaluator host.This is upstream bfcl-eval behavior, but this PR is what wires arbitrary-endpoint HTTP responses into it.
Options:
tc["name"]against the entry's known tool names inprocess_responsebefore handing calls to bfcl-eval.Deferred out of PR #346 to keep it focused; the multi-turn live-execution path is opt-in (its own CLI).