Dataset · AxiomicLabs
Theory of mind benchmark for small language models
AxiomicLabs/Tiny_Theory_of_Mind
English test set for evaluating the ability of small base models to understand others' intentions and beliefs.
What's inside
Contains between 1 and 10 thousand English question-answering test tasks with four multiple-choice options (random guessing gives 25%). The tasks cover various Theory of Mind topics and are divided by difficulty levels ranging from preschool to 6th grade. The benchmark is designed to evaluate base models using log-likelihood without instructions or chains of thought.
What you can do with it
- Evaluate base (non-instruction) compact language models on context understanding
- Compare the ability of small LLMs to solve Theory of Mind tasks
- Investigate cognitive skills of models of various sizes
- Tasks
- Question answering
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 550
- Likes
- 49
How to load it
from datasets import load_dataset
ds = load_dataset("AxiomicLabs/Tiny_Theory_of_Mind")