Skip to content

Dataset · yale-nlp

Expert multi-discipline video understanding benchmark

yale-nlp/MMVU

A set of 1k–10k videos and questions to test multimodal models on specialized topics.

What's inside

The MMVU evaluation dataset was created by researchers (including the Yale NLP group) to test deep video understanding in specialized scientific and professional fields. It includes between 1 and 10 thousand video and text examples. The authors also published a paper, leaderboard, and evaluation code.

What you can do with it

  • Benchmarking multimodal LLMs (video-text-to-text) on complex expert tasks
  • Comparing models' logical reasoning capabilities based on video materials
Tasks
Video text → text
Size
1K–10K rows
Downloads this month
3.1K
Likes
60

How to load it

from datasets import load_dataset

ds = load_dataset("yale-nlp/MMVU")