feat: summarize when tokenlimit #3227

fengju0213 · 2025-09-30T14:09:49Z

Description

Context creator now preserves the system message and returns the remaining history strictly by
timestamp, removing all per-message token bookkeeping.
- Token-limit handling captures the full context, rolls back recent tool-call chains when
  necessary, swaps in an assistant summary, and records summary state (depth, new-record counts,
  last user input). This state blocks back-to-back summaries with negligible progress and caps
  retries at three, preventing summarize→retry loops—even when the immediate overflow comes from a
  tool call.
- Passing token_limit into ChatAgent now logs a deprecation warning and ignores the value; callers
  should control limits through model backend configuration.
Pending work:
1.clean up the modules related to token_limit and the token counter.
2.Add unit tests that cover input strings capable of triggering the token-limit path.

Checklist

Go over all the following points, and put an x in all the boxes that apply.

I have read the CONTRIBUTION guide (required)
I have linked this PR to an issue using the Development section on the right sidebar or by adding Fixes #issue-number in the PR description (required)
I have checked if any dependencies need to be added or updated in pyproject.toml and uv lock
I have updated the tests accordingly (required for a bug fix or a new feature)
I have updated the documentation if needed:
I have added examples if this is a new feature

If you are unsure about any of these, don't hesitate to ask. We are here to help!

coderabbitai · 2025-09-30T14:09:57Z

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch enhance_update_memory

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Wendong-Fan · 2025-10-05T09:07:29Z

convert to draft as this PR hasn't updated with the approach we discussed

fengju0213 · 2025-10-14T08:30:06Z

Context creator now preserves the system message and returns the remaining history strictly by
timestamp, removing all per-message token bookkeeping.

Token-limit handling captures the full context, rolls back recent tool-call chains when
necessary, swaps in an assistant summary, and records summary state (depth, new-record counts,
last user input). This state blocks back-to-back summaries with negligible progress and caps
retries at three, preventing summarize→retry loops—even when the immediate overflow comes from a
tool call.
Passing token_limit into ChatAgent now logs a deprecation warning and ignores the value; callers
should control limits through model backend configuration.
Pending work:
1.clean up the modules related to token_limit and the token counter.
2.Add unit tests that cover input strings capable of triggering the token-limit path.

hesamsheikh · 2025-10-14T09:54:43Z

Thanks for the thorough PR @fengju0213. I wonder if the new implementation would cause inconsistencies and fragmentation of the summarization logic already implemented in two places.

ChatAgent.summarize() so the user can explicitly call to save the summarization.

summary_result = agent.summarize(filename="meeting_notes")
# Returns: {"summary": str, "file_path": str, "status": str}

This is mostly used to create workflow.md files of a session

ContextSummarizerToolkit user "USER" as the rolename, whereas here "ASSISTANT" is used.

fengju0213 · 2025-10-15T06:18:19Z

Thanks for the thorough PR @fengju0213. I wonder if the new implementation would cause inconsistencies and fragmentation of the summarization logic already implemented in two places.

ChatAgent.summarize() so the user can explicitly call to save the summarization.
summary_result = agent.summarize(filename="meeting_notes")
# Returns: {"summary": str, "file_path": str, "status": str}
This is mostly used to create workflow.md files of a session

ContextSummarizerToolkit user "USER" as the rolename, whereas here "ASSISTANT" is used.

thanks for reviewing！ @hesamsheikh We can probably unify the naming after rolename later. For now, could you help review and summarize the current logic for handling the token limit?

MuggleJinx

Thanks @fengju0213!! Left some comments. Maybe should also consider adding a test case when SUMMARY_MAX_DEPTH is reached.

camel/agents/chat_agent.py

Wendong-Fan · 2025-10-21T09:59:29Z

token counter

context 1,000,000 token

context/2 -> 500,000 token 10s

compress -> summarized memeory ( 100,000 token)
....

(1,000,000 -100,000) /2 +100,000 -> 550,000

compress -> summarized memeory ( 100,000 token)

(1,000,000 -100,000-100,000)) /2 +100,000 +100,000 -> 600,000
....

need to set the minium value for content compression

hesamsheikh

Thanks for the PR @fengju0213. I have added a few comments. As @MuggleJinx mentioned, i also think a few more test cases are a good idea.

camel/memories/context_creators/score_based.py

camel/agents/chat_agent.py

hesamsheikh · 2025-10-23T09:41:27Z

camel/agents/chat_agent.py

+            summary_msg = BaseMessage.make_assistant_message(
+                role_name="Assistant", content=content
+            )


here Assistant is used as role_name while in the ContextSummarizerToolkit USER role is used. Are you sure adding the summary as Assistant message is the best approach?

actually，i haven't compared two method in detail，do you have any suggestion？

hesamsheikh · 2025-10-23T09:46:28Z

camel/agents/chat_agent.py

+        # Add new summary
+        new_summary_msg = BaseMessage.make_assistant_message(
+            role_name="Assistant",
+            content=summary_with_prefix + " " + summary_status,


why is summary_status added directly to the summary? seems like a big.

will fix，should be file_path

hesamsheikh · 2025-10-23T09:54:17Z

camel/agents/chat_agent.py

+            for msg in messages:
+                content = msg.get('content', '')
+                if isinstance(content, str) and content.startswith(
+                    '[CAMEL_SUMMARY]'


I don't think [CAMEL_SUMMARY] allone is a good semantic prefix for the agent's summary.

how about CONTEXT_SUMMARY

If we want to actually send this to the LLM it must be more semantic, explaining what this summary is and how to treat it. For example: "The following is a summary of our conversation from a previous session, ...."

hesamsheikh · 2025-10-23T12:14:37Z

camel/agents/chat_agent.py

+                                f"Performing full compression."
+                            )
+                            # Summarize everything (including summaries)
+                            summary = self.summarize(include_summaries=True)


it's actually best we don't rely on the default prompt which is too generic. maybe mentioning why this summarization is happening would be a good idea. especially when using the ASSISTANT as the role the tone must be adjusted, and the default prompt doesn't do that.

I've ran a simple script to see how the summarization turns out.

CONTEXT BEFORE SUMMARIZATION (13 messages, 525 tokens)

Message 1 [SYSTEM]: You are a helpful assistant.
Message 2 [USER]: Hi! I'm working on a Python project and need help with data analysis.
Message 3 [ASSISTANT]: Hello! I'd be happy to help you with data analysis in Python...
Message 4 [USER]: I have a CSV file with customer sales data...
Message 5 [ASSISTANT]: Great! That's a common dataset structure. To analyze this data, we can use pandas...
Message 6 [USER]: Ok loaded it. Now I need to find the top 5 customers by total revenue.
Message 7 [ASSISTANT]: Perfect! To find the top 5 customers by revenue, we need to: 1. Calculate revenue...
Message 8 [USER]: That worked! But I noticed some customer IDs appear multiple times. Is that normal?
Message 9 [ASSISTANT]: Yes, that's completely normal! Each row represents a single transaction...
Message 10 [USER]: Makes sense. Can you also help me create a bar chart of the top 5 customers?
Message 11 [ASSISTANT]: Absolutely! We can use matplotlib for that...
Message 12 [USER]: Perfect! One more thing - how do I save this chart to a file?
Message 13 [ASSISTANT]: You can save the plot using plt.savefig() before plt.show()...

CONTEXT AFTER SUMMARIZATION (3 messages, 228 tokens)

Message 1 [SYSTEM]:
You are a helpful assistant.

Message 2 [ASSISTANT]:
[CAMEL_SUMMARY] - Dataset: CSV file with columns: customer_id, product_name, quantity, price

Tools used: pandas for data manipulation, matplotlib for visualization

Key actions:

Loaded CSV data using pd.read_csv()

Calculated revenue per row as quantity * price

Grouped data by customer_id and summed revenue to find total revenue per customer

Sorted and selected top 5 customers by total revenue

Created a bar chart to visualize top 5 customers' revenue using matplotlib

Saved the bar chart to a PNG file with high resolution and proper layout

Clarification: Multiple entries per customer_id are expected as each row is a separate transaction

Next steps / Action items:

Use provided code snippets to perform analysis and visualization

Save charts as image files for reporting or presentation success
^^^^^^^ BUG!

Message 3 [USER]:
Thanks! Can you also show me how to add a legend to the chart?

hesamsheikh · 2025-10-24T13:17:18Z

Hey @fengju0213 just as a reference that might be helpful, I ran Claude Code until it summarized the full context, and here is how it looks:

This session is being continued from a previous conversation that ran out of context. The conversation is summarized below:
Analysis:
Let me chronologically analyze this conversation about improving the xxx:

Summary:
1. **Primary Request and Intent**:
   [list]

2. **Key Technical Concepts**:
   [list]

3. **Files and Code Sections**:
[list]

4. **Errors and Fixes**:
[list]

5. **Problem Solving**:
   [list]

6. **All User Messages**:
   - "in the landing page put the brain more to the right, and space out the two bands a bit more to make room for the left card"
   - "the bands and the left card are not symmetric, the bottom band is too close to it"
   - "as the mouse moves, the position of bands changes too. don't do that."
   - etc

7. **Pending Tasks**:
   - None explicitly stated

8. **Current Work**:
   

9. **Optional Next Step**:
   xxxx
Please continue the conversation from where we left it off without asking the user any further questions. Continue with the last task that you were asked to work on.

I think there are helpful patterns here we can learn from. moreover, if you notice the tone of the summary (like "Continue with the last task...") you notice the role cannot be Assistant here. I think the best approach would be to append the summary to the system message and wipe the memory.

fengju0213 added 2 commits September 30, 2025 21:53

update

255670b

Update chat_agent.py

b1f3213

Update oceanbase.py

746a4f5

fengju0213 mentioned this pull request Oct 1, 2025

[BUG] Google calendar "XError: Failed to build service: No modulenamed 'google_auth_oauthlib'" eigent-ai/eigent#418

Closed

fengju0213 and others added 4 commits October 1, 2025 11:01

update

cc31148

Merge branch 'master' into enhance_update_memory

326543d

Merge branch 'master' into enhance_update_memory

aeffc5e

minor enhance

0dc9c12

Wendong-Fan marked this pull request as draft October 5, 2025 09:07

fengju0213 added 6 commits October 5, 2025 23:47

Merge branch 'master' into enhance_update_memory

40d404a

Update chat_agent.py

2df88da

Merge branch 'master' into enhance_update_memory

bd18fec

Update chat_agent.py

8c76dc5

Update chat_agent.py

cbdb4ef

update

b69fd16

fengju0213 changed the title ~~feat: retry when tokenlimit and enhance chunking logic~~ feat: summarize when tokenlimit Oct 14, 2025

Wendong-Fan assigned fengju0213 Oct 14, 2025

Wendong-Fan marked this pull request as ready for review October 14, 2025 08:32

Wendong-Fan requested a review from hesamsheikh October 14, 2025 08:32

fengju0213 added 2 commits October 14, 2025 16:33

Merge branch 'master' into enhance_update_memory

fc74702

Update chat_agent.py

ce3608e

MuggleJinx requested changes Oct 16, 2025

View reviewed changes

camel/agents/chat_agent.py Outdated Show resolved Hide resolved

camel/agents/chat_agent.py Show resolved Hide resolved

camel/agents/chat_agent.py Show resolved Hide resolved

camel/agents/chat_agent.py Outdated Show resolved Hide resolved

fengju0213 added 2 commits October 21, 2025 16:17

update

d184944

Update chat_agent.py

bc69be7

fengju0213 force-pushed the enhance_update_memory branch from b1885f3 to bc69be7 Compare October 21, 2025 09:53

fengju0213 force-pushed the enhance_update_memory branch from 97c5376 to bc69be7 Compare October 21, 2025 10:21

fengju0213 added 7 commits October 21, 2025 18:28

Merge branch 'master' into enhance_update_memory

c1e137e

Update chat_agent.py

3309c0f

Update chat_agent.py

8d66d0d

Update chat_agent.py

b0752ef

Update chat_agent.py

cc983c9

Update test_chat_agent.py

41ed7ee

Update test_chat_agent.py

71d367d

fengju0213 requested a review from MuggleJinx October 22, 2025 04:05

hesamsheikh requested changes Oct 23, 2025

View reviewed changes

fengju0213 added 2 commits October 23, 2025 21:44

Merge branch 'master' into enhance_update_memory

f2940e6

Update test_score_based.py

b45d558

Wendong-Fan added the Waiting for Update PR has been reviewed, need to be updated based on review comment label Oct 23, 2025

fengju0213 added 2 commits October 24, 2025 16:03

Update chat_agent.py

b5a21f1

Update chat_agent.py

3810c56

Merge branch 'master' into enhance_update_memory

2e78887

Uh oh!

feat: summarize when tokenlimit #3227

Are you sure you want to change the base?

feat: summarize when tokenlimit #3227

Conversation

fengju0213 commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

coderabbitai bot commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

Wendong-Fan commented Oct 5, 2025

Uh oh!

fengju0213 commented Oct 14, 2025

Uh oh!

hesamsheikh commented Oct 14, 2025

Uh oh!

fengju0213 commented Oct 15, 2025

Uh oh!

MuggleJinx left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Wendong-Fan commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hesamsheikh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

CONTEXT BEFORE SUMMARIZATION (13 messages, 525 tokens)

CONTEXT AFTER SUMMARIZATION (3 messages, 228 tokens)

Uh oh!

hesamsheikh commented Oct 24, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

fengju0213 commented Sep 30, 2025 •

edited

Loading

coderabbitai bot commented Sep 30, 2025 •

edited

Loading

MuggleJinx left a comment •

edited

Loading

Wendong-Fan commented Oct 21, 2025 •

edited

Loading