Skip to content

Commit e7dfed7

Browse files
savioloboclaude
andcommitted
feat: add 10 diverse fixture items for agent unit testing
- 5 simple items (card_swallowed, change_pin, terminate_account, top_up_by_cash_or_cheque, lost_or_stolen_card) - 5 ambiguous items on known confusion boundaries (unable_to_verify_identity, Refund_not_showing_up, pending_transfer, card_payment_not_recognised, why_verify_identity) - Each has query_id, text, true_intent, difficulty - All query_ids verified against holdout.parquet Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 760a71a commit e7dfed7

10 files changed

Lines changed: 60 additions & 0 deletions

File tree

tests/fixtures/items/item_001.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"query_id": "test_01938",
3+
"text": "What do I do if the ATM ate my card?",
4+
"true_intent": "card_swallowed",
5+
"difficulty": "simple"
6+
}

tests/fixtures/items/item_002.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"query_id": "test_02145",
3+
"text": "I need to change my PIN.",
4+
"true_intent": "change_pin",
5+
"difficulty": "simple"
6+
}

tests/fixtures/items/item_003.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"query_id": "test_01901",
3+
"text": "Can you delete my account please?",
4+
"true_intent": "terminate_account",
5+
"difficulty": "simple"
6+
}

tests/fixtures/items/item_004.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"query_id": "test_02441",
3+
"text": "How do I top-up using cash?",
4+
"true_intent": "top_up_by_cash_or_cheque",
5+
"difficulty": "simple"
6+
}

tests/fixtures/items/item_005.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"query_id": "test_00468",
3+
"text": "My card got lost.",
4+
"true_intent": "lost_or_stolen_card",
5+
"difficulty": "simple"
6+
}

tests/fixtures/items/item_006.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"query_id": "test_01203",
3+
"text": "I am having trouble verifying my identity.",
4+
"true_intent": "unable_to_verify_identity",
5+
"difficulty": "ambiguous"
6+
}

tests/fixtures/items/item_007.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"query_id": "test_01784",
3+
"text": "I requested a refund from a store but it hasn't arrived.",
4+
"true_intent": "Refund_not_showing_up",
5+
"difficulty": "ambiguous"
6+
}

tests/fixtures/items/item_008.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"query_id": "test_01879",
3+
"text": "How long does it take for a money transfer to show?",
4+
"true_intent": "pending_transfer",
5+
"difficulty": "ambiguous"
6+
}

tests/fixtures/items/item_009.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"query_id": "test_01110",
3+
"text": "I see a charge on my account that I don't recall making. I feel like my account may have been compromised.",
4+
"true_intent": "card_payment_not_recognised",
5+
"difficulty": "ambiguous"
6+
}

tests/fixtures/items/item_010.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
{
2+
"query_id": "test_01170",
3+
"text": "I would like to not have to do the identity verification.",
4+
"true_intent": "why_verify_identity",
5+
"difficulty": "ambiguous"
6+
}

0 commit comments

Comments
 (0)