Skip to content

Evaluation: Inconsistent Golden Q&As in Langfuse #449

@AkhileshNegi

Description

@AkhileshNegi

Describe the bug
When uploading a golden set of Q&As with a duplication factor in Langfuse, some question–answer pairs get mismatched. In a few cases, one question ends up paired with a different answer.

To Reproduce
There are no clear steps to reproduce this. The issue shows up sporadically.

Expected behavior
The dataset items in Langfuse should match the golden Q&A set exactly, when passed with duplication factor

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

Status

Closed

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions