Dataset · lukaemon
BIG-bench Hard: complex tasks for LLMs
lukaemon/bbh
The BIG-bench Hard benchmark with difficult tasks and reasoning chains for evaluating LLMs.
What's inside
This is a set of text tasks from BIG-bench Hard designed to test model reasoning. The card references the Suzgun et al. 2022 paper on tasks that test chain-of-thought. Size ranges from 1K to 10K examples; language is not specified in the metadata. The data is used as a benchmark.
What you can do with it
- Evaluating LLMs on complex reasoning tasks
- Testing chain-of-thought and few-shot approaches
- Comparing models on BIG-bench Hard
- Size
- 1K–10K rows
- Downloads this month
- 51.9K
- Likes
- 94
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("lukaemon/bbh")