feat: add Turkish Clinical Source Support evaluation task (clinical_safety group)#3903
Open
goktugozkanmd wants to merge 1 commit into
Open
Conversation
|
|
…afety group) - New clinical_safety group with turkish_clinical_source_support task - 10 Turkish-language clinical multiple-choice questions - Topics: diabetes, anticoagulation, COPD, contrast nephropathy, febrile neutropenia, hyperkalemia, metformin safety, DOAC management, opioid constipation, ACEi monitoring - Includes comprehensive boundary statement in README Closes EleutherAI#3866
goktugozkanmd
force-pushed
the
feat/turkish-clinical-source-support
branch
from
ed15aa6 to
1db2dbf
Compare
Author
|
Revalidated this against current
The branch is current and has no open review threads. @baberabb @0xSMT, could one of you review when you have bandwidth? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
clinical_safetytask group.turkish_clinical_source_support, a 10-item Turkish multiple-choice clinical safety task.Relation to issue
Closes #3866
Validation
.venv/bin/python -m lm_eval --model dummy --tasks turkish_clinical_source_support --limit 1 --verbosity ERROR --output_path /tmp/lm_eval_turkish_clinical_testSafety boundary