Skip to content

Dataset · nyu-mll

English grammar benchmark for LLMs

nyu-mll/blimp

67 thousand English minimal sentence pairs for testing grammar knowledge in language models.

What's inside

The BLiMP dataset includes 67 subsets of 1,000 minimal English sentence pairs, totaling 10K to 100K examples. Each pair isolates a specific difference in syntax, morphology, or semantics. Examples are automatically generated according to grammar rules compiled by experts.

What you can do with it

  • Evaluating linguistic and grammatical capabilities of language models
  • Comparing the quality of pre-trained LLMs on subtle syntactic distinctions
Tasks
Text classification
Languages
English
Size
10K–100K rows
Downloads this month
45.5K
Likes
49

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("nyu-mll/blimp")