Skip to content

Dataset · ArtificialAnalysis

Audio Version of Big Bench Hard

ArtificialAnalysis/big_bench_audio

1,000 English audio recordings with complex logical reasoning tasks for evaluating multimodal models.

What's inside

The dataset is an audio-recorded version of selected Big Bench Hard questions in English. It contains 1,000 audio recordings and texts across several categories, including formal logical fallacies and syllogisms (250 questions). The material was prepared by the Artificial Analysis team to evaluate audio-capable models.

What you can do with it

  • Check logical reasoning in neural networks with audio input support (audio-to-audio and multimodal LLMs)
  • Test voice assistant robustness to complex phrasing without prior transcription
Tasks
Audio → audio
Languages
English
Size
1K–10K rows
Downloads this month
15.5K
Likes
39

How to load it

from datasets import load_dataset

ds = load_dataset("ArtificialAnalysis/big_bench_audio")