Skip to content

[Fix] Video-MME-v2: improve acc calculation and data preparation#1551

Open
EliYuan30 wants to merge 1 commit into
open-compass:mainfrom
EliYuan30:fix/video-mme-v2-update
Open

[Fix] Video-MME-v2: improve acc calculation and data preparation#1551
EliYuan30 wants to merge 1 commit into
open-compass:mainfrom
EliYuan30:fix/video-mme-v2-update

Conversation

@EliYuan30
Copy link
Copy Markdown

This PR improves Video-MME-v2 support by:

  • making dataset preparation resumable when extracted videos are missing or incomplete
  • validating extracted video file sizes against zip metadata
  • adding progress reporting for Video-MME-v2 video preparation
  • defaulting Video-MME-v2 evaluation to exact matching when no judge is provided
  • adding accuracy results alongside group-level rating results

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant