Skip to content

Dataset · lighteval

PIQA physical commonsense benchmark

lighteval/piqa

An English question-answering dataset (10K–100K examples) to evaluate language models' understanding of everyday physics.

What's inside

The popular Physical Interaction: Question Answering (PIQA) English benchmark contains between 10,000 and 100,000 multiple-choice text tasks. The tasks test the understanding of basic physical properties of objects and everyday real-world interactions that are obvious to humans but challenging for language models.

What you can do with it

  • Evaluating logic and physical commonsense reasoning in large language models
  • Fine-tuning or SFT of models to improve answers to practical everyday questions
  • Comparing text models on a standard academic benchmark
Tasks
Question answering
Languages
English
Size
10K–100K rows
Downloads this month
25.6K
Likes
1

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("lighteval/piqa")