Dataset · nyu-mll
English grammar benchmark for LLMs
nyu-mll/blimp
67 thousand English minimal sentence pairs for testing grammar knowledge in language models.
What's inside
The BLiMP dataset includes 67 subsets of 1,000 minimal English sentence pairs, totaling 10K to 100K examples. Each pair isolates a specific difference in syntax, morphology, or semantics. Examples are automatically generated according to grammar rules compiled by experts.
What you can do with it
- Evaluating linguistic and grammatical capabilities of language models
- Comparing the quality of pre-trained LLMs on subtle syntactic distinctions
- Tasks
- Text classification
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 45.5K
- Likes
- 49
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("nyu-mll/blimp")