Skip to content

Dataset · Idavidrein

Challenging expert-level multiple-choice science questions

Idavidrein/gpqa

Expert-level multiple-choice questions in biology, physics, and chemistry to test complex scientific reasoning.

What's inside

GPQA is a set of multiple-choice questions written and reviewed by experts in biology, physics, and chemistry. The questions are very difficult: when answering outside their field, experts show only 34% accuracy, even with access to Google for over 30 minutes. The language is English, the size ranges from 1k to 10k records; the card requests not to publish examples openly to prevent leakage into training corpora.

What you can do with it

  • Evaluating LLMs on scientific reasoning benchmarks
  • Assessing models in multiple-choice QA
  • Checking for dataset leakage in training corpora
Tasks
Question answering, Text generation
Languages
English
Size
1K–10K rows
Downloads this month
109.4K
Likes
692
Access
on request on Hugging Face

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("Idavidrein/gpqa")