Skip to content

Dataset · EleutherAI

LAMBADA multilingual text understanding benchmark

EleutherAI/lambada_openai

Classic test for predicting the last word in a passage across five European languages.

What's inside

The dataset is a test sample of LAMBADA in OpenAI preprocessing, supplemented with machine translations into German, Spanish, French, and Italian in addition to the original English. It contains between 10 and 100 thousand short literary texts where it is easy for a human to guess the final word from the general meaning of the passage. Used to test how well language models capture broad context.

What you can do with it

  • Evaluating LLM ability to understand long text context in 5 languages
  • Comparing language model quality on a standard word-prediction benchmark
Languages
German, English, Spanish, French, Italian
Size
10K–100K rows
Downloads this month
120.9K
Likes
56

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("EleutherAI/lambada_openai")