Dataset · yale-nlp
Expert multi-discipline video understanding benchmark
yale-nlp/MMVU
A set of 1k–10k videos and questions to test multimodal models on specialized topics.
What's inside
The MMVU evaluation dataset was created by researchers (including the Yale NLP group) to test deep video understanding in specialized scientific and professional fields. It includes between 1 and 10 thousand video and text examples. The authors also published a paper, leaderboard, and evaluation code.
What you can do with it
- Benchmarking multimodal LLMs (video-text-to-text) on complex expert tasks
- Comparing models' logical reasoning capabilities based on video materials
- Tasks
- Video text → text
- Size
- 1K–10K rows
- Downloads this month
- 3.1K
- Likes
- 60
How to load it
from datasets import load_dataset
ds = load_dataset("yale-nlp/MMVU")