feat: Support Multiple System Messages with Better API

[OmCheeLin]

Description

Support Multiple System Messages with Better API. #3141

Checklist

Go over all the following points, and put an x in all the boxes that apply.

• I have read the CONTRIBUTION guide (required)
• I have linked this PR to an issue using the Development section on the right sidebar or by adding Fixes #issue-number in the PR description (required)
• I have checked if any dependencies need to be added or updated in pyproject.toml and uv lock
• I have updated the tests accordingly (required for a bug fix or a new feature)
• I have updated the documentation if needed:
• I have added examples if this is a new feature

If you are unsure about any of these, don't hesitate to ask. We are here to help!

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch better-api

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Wendong-Fan]

thanks @OmCheeLin 's contribution! based on previous discussion: #3113 (comment) I think we also need to update ScoreBasedContextCreator to fully support multiple system message setting

[hesamsheikh]

Thanks for the PR @OmCheeLin
To make the PR comprehensive, these missing parts also need to be addressed:

1. adding test cases
2. the method add_or_update_system_messages doesn't seem to implement the multiple system messages we are looking for. It must be able to create more than one system message, even in the middle of the conversation.

[Wendong-Fan]

hi @OmCheeLin ,

Hope you're doing well!

Just wanted to check in on this PR. It looks like there's some feedback that needs to be addressed, along with some failing in pre-commit checks.

Please let us know if you have any questions or need a hand with anything. We're looking forward to move this feature forward with you!

Cheers

[OmCheeLin]

I have asked hesamsheikh in the issue before, but he hasn't replied to me yet. Of course, this issue can be given to others if anyone wants to do it.

[hesamsheikh]

Hey @OmCheeLin

I haven't seen any questions asked here or on Slack. Have I missed anything?

I have already given a couple feedbacks, and you removed the "review required" tag. If you need assistance or clarifications, let me know.

Thank you.

[OmCheeLin]

Hey @OmCheeLin

I haven't seen any questions asked here or on Slack. Have I missed anything?

I have already given a couple feedbacks, and you removed the "review required" tag. If you need assistance or clarifications, let me know.

Thank you.

oh, I asked in #3141, I will send messages under pr in the future

[hesamsheikh]

Thanks for pointing it out @OmCheeLin
Yeah, it's best to keep the discussions in the PR for the future. I will take a look and respond to it shortly.

[hesamsheikh]

Hey @OmCheeLin

Following the discussions in the issue #3141 , let's now focus on making the API more flexible and intuitive and then later on, work on having multiple system messages. I think the best approach for now would be to imrprove the following areas:

1. Add make_system_message() factory method. No need to support video and image for system messages at this point. We can add this method to BaseMessage class instead of calling the make_assistant_message() method.
2. to make it more flexible to update the system message, it would be a good idea to add system message update methods to ChatAgent which can be done in two methods:

• update_system_message()
• Replace the entire system message with new content
• Update both _original_system_message and _system_message
• Call init_messages() to reset conversation with new system message
  or,
• append_to_system_message()
• Append additional context to existing system message
• Accept str for content to append
• Update both _original_system_message and _system_message
• Call init_messages() to update memory

For now this is more needed and also an easier to implement enhancement than the multi system messages. To be honest, I'm not even sure we would need to have multiple system messages if we have the append_to_system_message and update_system_message since this gives us huge flexibility. But this is a discussion for later, which @Wendong-Fan can weigh in. Please let me know if you need any assistance.

[OmCheeLin]

@hesamsheikh I agree with your point of view. If @Wendong-Fan agrees as well, I can start working.

[Wendong-Fan]

Hey @OmCheeLin

Following the discussions in the issue #3141 , let's now focus on making the API more flexible and intuitive and then later on, work on having multiple system messages. I think the best approach for now would be to imrprove the following areas:

1. Add make_system_message() factory method. No need to support video and image for system messages at this point. We can add this method to BaseMessage class instead of calling the make_assistant_message() method.
2. to make it more flexible to update the system message, it would be a good idea to add system message update methods to ChatAgent which can be done in two methods:

• update_system_message()
• Replace the entire system message with new content
• Update both _original_system_message and _system_message
• Call init_messages() to reset conversation with new system message
  or,
• append_to_system_message()
• Append additional context to existing system message
• Accept str for content to append
• Update both _original_system_message and _system_message
• Call init_messages() to update memory

For now this is more needed and also an easier to implement enhancement than the multi system messages. To be honest, I'm not even sure we would need to have multiple system messages if we have the append_to_system_message and update_system_message since this gives us huge flexibility. But this is a discussion for later, which @Wendong-Fan can weigh in. Please let me know if you need any assistance.

thanks @OmCheeLin and @hesamsheikh ! Let's add update_system_message and append_to_system_message first

[hesamsheikh]

Thanks for the fast implementation @OmCheeLin
I added a few comments. I'm not sure if it is necessary to reset the memory after appending to or updating the system message. At that point, maybe it makes more sense to initialize the agent again? Wdyt?

[hesamsheikh]

let's put some kind of a separator between the existing and the new system message. like a newline.

[hesamsheikh]

let's add a validation for case that None is passed (like return)

[OmCheeLin]

Thanks for the fast implementation @OmCheeLin I added a few comments. I'm not sure if it is necessary to reset the memory after appending to or updating the system message. At that point, maybe it makes more sense to initialize the agent again? Wdyt?

Resetting the entire system means clearing the output format, language, tools and etc. Do I need to do this?

[hesamsheikh]

Thanks for the fast implementation @OmCheeLin I added a few comments. I'm not sure if it is necessary to reset the memory after appending to or updating the system message. At that point, maybe it makes more sense to initialize the agent again? Wdyt?

Resetting the entire system means clearing the output format, language, tools and etc. Do I need to do this?

What I mean is, you reset the memory after updating or appending to the system message. let's add an argument to those methods to make it customizable whether the memory must be reset or just continue the conversation.