Dataset · FBK-MT
Multilingual multimodal benchmark on scientific reports
FBK-MT/MCIF
Human-annotated instruction dataset based on scientific reports: speech, video, and text across four languages to evaluate MLLMs.
What's inside
MCIF is a human-annotated multilingual benchmark for evaluating instruction following in cross-lingual and multimodal scenarios over short and long inputs. It includes speech, images/video, and text in English, German, Italian, and Chinese. Size ranges from 1,000 to 10,000 examples; tasks include ASR, QA, summarization, VQA, and translation.
What you can do with it
- Evaluate MLLMs on cross-lingual instructions with audio and video
- Assess ASR, summarization, and VQA in a single multimodal pipeline
- Benchmark instruction models on scientific reports
- Tasks
- Automatic speech recognition, Question answering, Summarization, Visual question answering, Translation
- Languages
- English, German, Italian, Chinese
- Size
- 1K–10K rows
- Downloads this month
- 3.2K
- Likes
- 71
How to load it
from datasets import load_dataset
ds = load_dataset("FBK-MT/MCIF")