Skip to content

Dataset · AxiomicLabs

Theory of mind benchmark for small language models

AxiomicLabs/Tiny_Theory_of_Mind

English test set for evaluating the ability of small base models to understand others' intentions and beliefs.

What's inside

Contains between 1 and 10 thousand English question-answering test tasks with four multiple-choice options (random guessing gives 25%). The tasks cover various Theory of Mind topics and are divided by difficulty levels ranging from preschool to 6th grade. The benchmark is designed to evaluate base models using log-likelihood without instructions or chains of thought.

What you can do with it

  • Evaluate base (non-instruction) compact language models on context understanding
  • Compare the ability of small LLMs to solve Theory of Mind tasks
  • Investigate cognitive skills of models of various sizes
Tasks
Question answering
Languages
English
Size
1K–10K rows
Downloads this month
550
Likes
49

How to load it

from datasets import load_dataset

ds = load_dataset("AxiomicLabs/Tiny_Theory_of_Mind")