Skip to content

fix(bedrock-kb-retrieval): preserve non-ASCII text in tool results - #3821

Open
kimnamu wants to merge 2 commits into
awslabs:mainfrom
kimnamu:fix/bedrock-kb-non-ascii
Open

fix(bedrock-kb-retrieval): preserve non-ASCII text in tool results#3821
kimnamu wants to merge 2 commits into
awslabs:mainfrom
kimnamu:fix/bedrock-kb-non-ascii

Conversation

@kimnamu

@kimnamu kimnamu commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Thank you for maintaining the AWS MCP servers.

Fixes #3820

Summary

QueryKnowledgeBases and ListKnowledgeBases serialized results with json.dumps(...) (default ensure_ascii=True), so non-ASCII KB content (CJK, accented, emoji) was returned as \uXXXX escapes — token bloat + unreadable payloads. This adds ensure_ascii=False to the two user/model-visible serialization points (the MCP transport is already UTF-8).

Changes

  • knowledgebases/retrieval.py:113, server.py:128 — add ensure_ascii=False
  • regression tests in test_retrieval.py + test_server.py
  • ## [Unreleased] / ### Fixed CHANGELOG entry

Before / After

Item Before After
KB text 東京は日本の首都です。 in tool output 東京... (escaped) 東京は日本の首都です。
KB name 한국어 데이터 소스 한국... 한국어 데이터 소스
Output is valid JSON / round-trips via json.loads ✅ (unchanged)
Public API / tool signatures / result ordering ✅ Unchanged
ASCII-only content ✅ identical bytes ✅ identical bytes

Tests

40 passed, ruff check clean. Reverting the source change (keeping the tests) makes the 2 new tests FAIL with assert '東京...' in '...\\u6771\\u4eac...', proving they catch the bug; restoring it → all pass.


Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the project license.

Knowledge Base content containing CJK / accented / emoji characters was
serialized with json.dumps(...) using the default ensure_ascii=True, so
QueryKnowledgeBases and ListKnowledgeBases returned \uXXXX escape
sequences instead of readable text. This bloats tokens and makes the
content unreadable to both the model and the user.

Serialize with ensure_ascii=False in the two model/user-visible
json.dumps calls. Add regression tests asserting CJK content survives
round-trip without \u escaping.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kimnamu

kimnamu commented Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

Status update (no action needed — just keeping this current):

Happy to rebase or adjust anything (e.g. drop the CHANGELOG entry) if preferred — thanks for your time reviewing!

sergioperezcheco added a commit to sergioperezcheco/mcp that referenced this pull request Jul 17, 2026
…luate

json.dumps defaults to ensure_ascii=True, which escapes CJK characters,
emoji, and accented characters to \uXXXX sequences. This makes the
output unreadable for non-Latin scripts.

Add ensure_ascii=False so the result string preserves the original
Unicode characters, matching the fix already applied to
bedrock-kb-retrieval (PR awslabs#3821).

Fixes awslabs#3994
sergioperezcheco added a commit to sergioperezcheco/mcp that referenced this pull request Jul 18, 2026
…luate

json.dumps defaults to ensure_ascii=True, which escapes CJK characters,
emoji, and accented characters to \uXXXX sequences. This makes the
output unreadable for non-Latin scripts.

Add ensure_ascii=False so the result string preserves the original
Unicode characters, matching the fix already applied to
bedrock-kb-retrieval (PR awslabs#3821).

Fixes awslabs#3994
sergioperezcheco added a commit to sergioperezcheco/mcp that referenced this pull request Jul 20, 2026
…luate

json.dumps defaults to ensure_ascii=True, which escapes CJK characters,
emoji, and accented characters to \uXXXX sequences. This makes the
output unreadable for non-Latin scripts.

Add ensure_ascii=False so the result string preserves the original
Unicode characters, matching the fix already applied to
bedrock-kb-retrieval (PR awslabs#3821).

Fixes awslabs#3994
sergioperezcheco added a commit to sergioperezcheco/mcp that referenced this pull request Jul 21, 2026
…luate

json.dumps defaults to ensure_ascii=True, which escapes CJK characters,
emoji, and accented characters to \uXXXX sequences. This makes the
output unreadable for non-Latin scripts.

Add ensure_ascii=False so the result string preserves the original
Unicode characters, matching the fix already applied to
bedrock-kb-retrieval (PR awslabs#3821).

Fixes awslabs#3994
sergioperezcheco added a commit to sergioperezcheco/mcp that referenced this pull request Jul 21, 2026
…luate

json.dumps defaults to ensure_ascii=True, which escapes CJK characters,
emoji, and accented characters to \uXXXX sequences. This makes the
output unreadable for non-Latin scripts.

Add ensure_ascii=False so the result string preserves the original
Unicode characters, matching the fix already applied to
bedrock-kb-retrieval (PR awslabs#3821).

Fixes awslabs#3994
sergioperezcheco added a commit to sergioperezcheco/mcp that referenced this pull request Jul 22, 2026
…luate

json.dumps defaults to ensure_ascii=True, which escapes CJK characters,
emoji, and accented characters to \uXXXX sequences. This makes the
output unreadable for non-Latin scripts.

Add ensure_ascii=False so the result string preserves the original
Unicode characters, matching the fix already applied to
bedrock-kb-retrieval (PR awslabs#3821).

Fixes awslabs#3994
sergioperezcheco added a commit to sergioperezcheco/mcp that referenced this pull request Jul 22, 2026
…luate

json.dumps defaults to ensure_ascii=True, which escapes CJK characters,
emoji, and accented characters to \uXXXX sequences. This makes the
output unreadable for non-Latin scripts.

Add ensure_ascii=False so the result string preserves the original
Unicode characters, matching the fix already applied to
bedrock-kb-retrieval (PR awslabs#3821).

Fixes awslabs#3994
sergioperezcheco added a commit to sergioperezcheco/mcp that referenced this pull request Jul 23, 2026
…luate

json.dumps defaults to ensure_ascii=True, which escapes CJK characters,
emoji, and accented characters to \uXXXX sequences. This makes the
output unreadable for non-Latin scripts.

Add ensure_ascii=False so the result string preserves the original
Unicode characters, matching the fix already applied to
bedrock-kb-retrieval (PR awslabs#3821).

Fixes awslabs#3994
sergioperezcheco added a commit to sergioperezcheco/mcp that referenced this pull request Jul 24, 2026
…luate

json.dumps defaults to ensure_ascii=True, which escapes CJK characters,
emoji, and accented characters to \uXXXX sequences. This makes the
output unreadable for non-Latin scripts.

Add ensure_ascii=False so the result string preserves the original
Unicode characters, matching the fix already applied to
bedrock-kb-retrieval (PR awslabs#3821).

Fixes awslabs#3994
@github-actions

Copy link
Copy Markdown
Contributor

This pull request is now marked as stale because it hasn't seen activity for a while. Add a comment or it will be closed soon. If you wish to exclude this issue from being marked as stale, add the "backlog" label.

@github-actions github-actions Bot added the stale These are items that have been around for a long time without progress label Jul 26, 2026
@kimnamu

kimnamu commented Jul 26, 2026

Copy link
Copy Markdown
Contributor Author

Re-verified after the stale bump: rebased locally onto current main (clean, no conflicts) and ran the full suite for bedrock-kb-retrieval-mcp-server40 passed, ruff check clean. Both call sites (retrieval.py:113, server.py:128) still default to ensure_ascii=True, so the fix is still needed and unchanged from the original PR.

Still happy to adjust anything (e.g. drop the CHANGELOG entry) if that unblocks a review — thanks for keeping this open.

sergioperezcheco added a commit to sergioperezcheco/mcp that referenced this pull request Jul 26, 2026
…luate

json.dumps defaults to ensure_ascii=True, which escapes CJK characters,
emoji, and accented characters to \uXXXX sequences. This makes the
output unreadable for non-Latin scripts.

Add ensure_ascii=False so the result string preserves the original
Unicode characters, matching the fix already applied to
bedrock-kb-retrieval (PR awslabs#3821).

Fixes awslabs#3994
@github-actions github-actions Bot removed the stale These are items that have been around for a long time without progress label Jul 27, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This pull request is now marked as stale because it hasn't seen activity for a while. Add a comment or it will be closed soon. If you wish to exclude this issue from being marked as stale, add the "backlog" label.

@github-actions github-actions Bot added the stale These are items that have been around for a long time without progress label Aug 11, 2026
@kimnamu

kimnamu commented Aug 11, 2026

Copy link
Copy Markdown
Contributor Author

Still relevant as of today's main (ebcafaed):

Two things that may help a decision:

  1. The same one-line change was merged for a sibling server in fix(amazon-bedrock-agentcore-mcp-server): preserve non-ASCII text in browser_evaluate results #3995 (browser_evaluate, ensure_ascii=False), so the approach already has precedent in this repo.
  2. fix(bedrock-kb-retrieval): preserve non-ASCII characters in tool output #4372 proposes the same change to the same two lines. No need to review both — if that one is easier to take, I'll close this and bedrock-kb-retrieval: QueryKnowledgeBases/ListKnowledgeBases return non-ASCII (CJK, emoji) as \uXXXX escapes #3820 can point there instead.

If this is only waiting on review bandwidth, the backlog label would keep the stale bot off it. Thanks for your time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale These are items that have been around for a long time without progress

Projects

Status: To triage

Development

Successfully merging this pull request may close these issues.

bedrock-kb-retrieval: QueryKnowledgeBases/ListKnowledgeBases return non-ASCII (CJK, emoji) as \uXXXX escapes

1 participant