Skip to content

Dataset · lukaemon

BIG-bench Hard: complex tasks for LLMs

lukaemon/bbh

The BIG-bench Hard benchmark with difficult tasks and reasoning chains for evaluating LLMs.

What's inside

This is a set of text tasks from BIG-bench Hard designed to test model reasoning. The card references the Suzgun et al. 2022 paper on tasks that test chain-of-thought. Size ranges from 1K to 10K examples; language is not specified in the metadata. The data is used as a benchmark.

What you can do with it

  • Evaluating LLMs on complex reasoning tasks
  • Testing chain-of-thought and few-shot approaches
  • Comparing models on BIG-bench Hard
Size
1K–10K rows
Downloads this month
51.9K
Likes
94

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("lukaemon/bbh")