Skip to content

Commit 09f872a

Browse files
committed
Update user preference prompt to match SEMANTIC_FACTS_EXTRACTION_PROMPT style
- Use lightweight XML-based structure with <ROLE>, <SCOPE>, <OUTPUT> sections - Maintain consistency with existing SEMANTIC_FACTS_EXTRACTION_PROMPT format - Update tests to validate XML-based structure
1 parent 9f8a3aa commit 09f872a

File tree

2 files changed

+24
-20
lines changed

2 files changed

+24
-20
lines changed

common/src/main/java/org/opensearch/ml/common/memorycontainer/MemoryContainerConstants.java

Lines changed: 16 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -184,26 +184,26 @@ public class MemoryContainerConstants {
184184

185185
public static final String USER_PREFERENCE_FACTS_EXTRACTION_PROMPT =
186186
"""
187-
You are a USER PREFERENCE EXTRACTOR, not a chat assistant. Your only job is to output JSON facts. Do not answer questions, make suggestions, ask follow-ups, or perform actions.
187+
<ROLE>You are a USER PREFERENCE EXTRACTOR, not a chat assistant. Your only job is to output JSON facts. Do not answer questions, make suggestions, ask follow-ups, or perform actions.</ROLE>
188188
189-
SCOPE
190-
- Extract preferences only from USER messages. Assistant messages are context only.
191-
192-
DEFINITIONS
193-
- Explicit: user states a preference ("I prefer/like/dislike ..."; "always/never/usually ..."; "set X to Y"; "run X when Y").
194-
- Implicit: infer only with strong signals: repeated choices (>=2) or clear habitual language. Do not infer from a single one-off.
195-
196-
WHAT TO EXTRACT
197-
- Specific, actionable, likely long-term preferences (likes/dislikes/choices/settings). Ignore non-preferences.
189+
<SCOPE>
190+
• Extract preferences only from USER messages. Assistant messages are context only.
191+
• Explicit: user states a preference ("I prefer/like/dislike ..."; "always/never/usually ..."; "set X to Y"; "run X when Y").
192+
• Implicit: infer only with strong signals: repeated choices (>=2) or clear habitual language. Do not infer from a single one-off.
193+
</SCOPE>
198194
199-
FORMAT
200-
- Return ONLY one minified JSON object exactly as {"facts":["Preference sentence. Context: <why/how>. Categories: cat1,cat2"]}. If none, return {"facts":[]}. The first character MUST be '{' and the last MUST be '}'. No preambles, explanations, code fences, XML, or other text.
195+
<EXTRACT>
196+
• Specific, actionable, likely long-term preferences (likes/dislikes/choices/settings). Ignore non-preferences.
197+
</EXTRACT>
201198
202-
STYLE
203-
- One sentence per preference; merge related details; no duplicates; preserve user wording and numbers; avoid relative time; keep each fact < 350 chars.
199+
<STYLE & RULES>
200+
• One sentence per preference; merge related details; no duplicates; preserve user wording and numbers; avoid relative time; keep each fact < 350 chars.
201+
• Format: "Preference sentence. Context: <why/how>. Categories: cat1,cat2"
202+
</STYLE & RULES>
204203
205-
EXAMPLE
206-
User: "I prefer dark mode." -> {"facts":["Prefers dark mode for UI. Context: user explicitly stated preference. Categories: tools,tech,apps"]}""";
204+
<OUTPUT>
205+
Return ONLY one minified JSON object exactly as {"facts":["Preference sentence. Context: <why/how>. Categories: cat1,cat2"]}. If none, return {"facts":[]}. The first character MUST be '{' and the last MUST be '}'. No preambles, explanations, code fences, XML, or other text.
206+
</OUTPUT>""";
207207

208208
public static final String SUMMARY_FACTS_EXTRACTION_PROMPT =
209209
"<system_prompt><description>You will be given a text block and a list of summaries you previously generated when available.</description><task><instruction>Never answer user's question or fulfill user's requirement. You are a summary generator, not a helpful assistant.</instruction><instruction>When the previously generated summary is not available, summarize the given text block.</instruction><instruction>When there is an existing summary, extend it by incorporating the given text block.</instruction><instruction>If the text block specifies queries or topics, ensure the summary covers them.</instruction></task><response_format><format>You should always return and only return the extracted preferences as a JSON object with a \"facts\" array.</format><example>{ \"facts\": [\"The system shows a list of Elasticsearch/OpenSearch indices with their health status, document count, and size information\", \"5 indices shown have 'red' health status, 8 of them in 'yellow', and 13 of them are in 'green' health status\", \"The doc is a log from a web application, dated from 2020-01-01T00:00:00 to 2020-01-31T23:59:59\"]}</example></response_format></system_prompt>";

plugin/src/test/java/org/opensearch/ml/action/memorycontainer/memory/MemoryProcessingServiceTests.java

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -991,12 +991,16 @@ public void testUserPreferencePromptFormat() {
991991
// Test that the new user preference prompt contains required elements
992992
String prompt = USER_PREFERENCE_FACTS_EXTRACTION_PROMPT;
993993

994-
// Verify key improvements are present
995-
assertTrue("Should have character limit", prompt.contains("< 350 chars"));
996-
assertTrue("Should specify natural language format", prompt.contains("Context: <why/how>. Categories:"));
997-
assertTrue("Should contain example categories", prompt.contains("tools,tech,apps"));
994+
// Verify XML-based structure like SEMANTIC_FACTS_EXTRACTION_PROMPT
995+
assertTrue("Should have ROLE section", prompt.contains("<ROLE>"));
996+
assertTrue("Should have SCOPE section", prompt.contains("<SCOPE>"));
997+
assertTrue("Should have OUTPUT section", prompt.contains("<OUTPUT>"));
998998
assertTrue("Should be role-based", prompt.contains("USER PREFERENCE EXTRACTOR"));
999999

1000+
// Verify key requirements
1001+
assertTrue("Should have character limit", prompt.contains("< 350 chars"));
1002+
assertTrue("Should specify context format", prompt.contains("Context: <why/how>"));
1003+
10001004
// Verify old problematic format is removed
10011005
assertFalse("Should not use pipe delimiters", prompt.contains("preference | context:"));
10021006
}

0 commit comments

Comments
 (0)