Dataset · EleutherAI
LAMBADA multilingual text understanding benchmark
EleutherAI/lambada_openai
Classic test for predicting the last word in a passage across five European languages.
What's inside
The dataset is a test sample of LAMBADA in OpenAI preprocessing, supplemented with machine translations into German, Spanish, French, and Italian in addition to the original English. It contains between 10 and 100 thousand short literary texts where it is easy for a human to guess the final word from the general meaning of the passage. Used to test how well language models capture broad context.
What you can do with it
- Evaluating LLM ability to understand long text context in 5 languages
- Comparing language model quality on a standard word-prediction benchmark
- Languages
- German, English, Spanish, French, Italian
- Size
- 10K–100K rows
- Downloads this month
- 120.9K
- Likes
- 56
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("EleutherAI/lambada_openai")