Dataset · Idavidrein
Challenging expert-level multiple-choice science questions
Idavidrein/gpqa
Expert-level multiple-choice questions in biology, physics, and chemistry to test complex scientific reasoning.
What's inside
GPQA is a set of multiple-choice questions written and reviewed by experts in biology, physics, and chemistry. The questions are very difficult: when answering outside their field, experts show only 34% accuracy, even with access to Google for over 30 minutes. The language is English, the size ranges from 1k to 10k records; the card requests not to publish examples openly to prevent leakage into training corpora.
What you can do with it
- Evaluating LLMs on scientific reasoning benchmarks
- Assessing models in multiple-choice QA
- Checking for dataset leakage in training corpora
- Tasks
- Question answering, Text generation
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 109.4K
- Likes
- 692
- Access
- on request on Hugging Face
Models trained on it
- mradermacher/Qwen3-4B-Instruct-2507-20250808-233922-0-i1-GGUFText generation 1.3K
- mradermacher/Qwen3-4B-Instruct-2507-20250808-233922-0-GGUFText generation 618
- guray16/Qwen3-4B-Instruct-2507-20250808-233922-0-i1-GGUFText generation 608
- mradermacher/Qwen3.5-0.8B-Dark-Genius-GGUF 561
- EganAI/Qwen3-4B-Instruct-2507-20250808-233922-0-Q8_0-GGUFText generation 63
- DexinR/qwen2.5-math220k-ckpt15000Text generation 46
- pymlex/Qwen3.5-0.8B-Dark-GeniusText generation 34
- EganAI/Qwen3-4B-Instruct-2507-20250808-233922-0Text generation 28
How to load it
from datasets import load_dataset
ds = load_dataset("Idavidrein/gpqa")