Passing Reserve Keywords (kernel, chatHistory) to Functions with ChatCompletion Agents

[haithamshahin333]

Hello,

I'm noticing that with a ChatCompletion agent that has a plugin with functions that require the reserved keyword parameters are not recognizing that they should be intercepted from the LLM needing to identify those arguments (referenced here: #10214).

As an example, I defined a basic kernel function with chat_history as a parameter, expecting that this would not be serialized in the tool call definition to the LLM but that SK would pass this into the function's context. After some logging, it looks as though the LLM is extracting what should be passed like any other argument in the kernel function.

[moonbox3]

Hi @haithamshahin333,

Thanks for raising this question. Here’s a breakdown of how the ChatCompletionAgent currently handles reserved keyword parameters like chat_history and where we might improve clarity.

How It Works Today

With the chat completion agent, you can pass chat_history (or in my test case below, renamed as history) as part of the kernel arguments. This argument is used to generate the prompt for the model. In our implementation, we explicitly remove this argument from the kernel arguments before passing them to the function. (I acknowledge that this behavior could be cleaner.)

Example Test Kernel Function

For testing, I defined a kernel function that retrieves the chat history. Notice that I renamed the parameter to history and placed it within KernelArguments:

class History:
    @kernel_function(description="Provides the chat history.")
    def get_history(self, arguments: KernelArguments) -> Annotated[str, "Returns the chat history."]:
        chat_history = arguments.get("history")
        if chat_history:
            print(f"Chat History: {chat_history}")

Since this function requires the arguments parameter, a basic JSON Schema is generated and sent to the model. If you provide a default value for arguments (making it optional), we still include it in the JSON schema, but the model won’t be forced to supply it.

Parameter Handling in the Code

In the gather_function_parameters(...) method within KernelFunctionFromMethod, the code specifically checks for the parameter named arguments:

if param.name == "arguments":
    function_arguments[param.name] = context.arguments
    continue

This makes sure that if the arguments are provided and recognized as part of the KernelParameterMetadata, they are passed correctly to the kernel function.

JSON Schema Overview

For the example kernel function I showed above, the JSON Schema advertised to the model is structured as follows:

{
    "type": "function",
    "function": {
        "name": "History-get_history",
        "description": "Provides the chat history.",
        "parameters": {
            "type": "object",
            "properties": {
                "arguments": {
                    "type": "object",
                    "properties": {}
                }
            },
            "required": ["arguments"]
        }
    }
}

Potential Improvement

One improvement to consider is not advertising the arguments parameter in the JSON schema at all. We had a solution for this in the past, which was later removed to align with .NET. It appears that the .NET team has moved forward with handling it, while the Python implementation still awaits a similar update. I recognize that since some of the parameters are reserved, we don't need to send them, and thus can save on some token cost.

[haithamshahin333]

Thank you @moonbox3 appreciate the detailed response. To clarify, this also assumes that in your invoke or get_response call that you would be passing those reserved parameters explicitly correct? I am seeing the expected behavior above when I invoke my agent as follows:

chat_history=ChatHistory()
await agent.get_response(history=chat_history, arguments=KernelArguments(history=chat_history), kernel=kernel)

[moonbox3]

Yes, @haithamshahin333, that's how I was running the sample, too -- by providing the kernel args in the same way you are. One option, if you don't want to provide the kernel argument specifically as part of a kernel function, you could define the kernel as a class attribute, like this:

class History:
    def __init__(self, kernel: Kernel):
        self.kernel = kernel

    @kernel_function(description="Provides the chat history.")
    def get_history(self, arguments: KernelArguments) -> Annotated[str, "Returns the chat history."]:
        chat_history = arguments.get("history")
        if chat_history:
            print(f"Chat History: {chat_history}")
        # do something with self.kernel ...

For my understanding and clarity: are you hitting specific issues when proving the Kernel or ChatHistory as part of a kernel function, or you had understood it as: because they're reserved, they shouldn't need to be included as part of the JSON Schema?

[haithamshahin333]

Thank you @moonbox3 this is helpful validation.

On your question, it was the latter - I wasn't hitting any issues necessarily but just wanted to make sure I understood the expected behavior given they are reserved. I was confused since I assumed that because they are reserved and would be automatically injected by the framework that they wouldn't be serialized which made me think I was misunderstanding what was meant by "reserved" when they saw them in my debug output for what was being sent to the LLM.