Dataset · ArtificialAnalysis
Audio Version of Big Bench Hard
ArtificialAnalysis/big_bench_audio
1,000 English audio recordings with complex logical reasoning tasks for evaluating multimodal models.
What's inside
The dataset is an audio-recorded version of selected Big Bench Hard questions in English. It contains 1,000 audio recordings and texts across several categories, including formal logical fallacies and syllogisms (250 questions). The material was prepared by the Artificial Analysis team to evaluate audio-capable models.
What you can do with it
- Check logical reasoning in neural networks with audio input support (audio-to-audio and multimodal LLMs)
- Test voice assistant robustness to complex phrasing without prior transcription
- Tasks
- Audio → audio
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 15.5K
- Likes
- 39
How to load it
from datasets import load_dataset
ds = load_dataset("ArtificialAnalysis/big_bench_audio")