Skip to content

Dataset · EleutherAI

Full Wikipedia articles at the document level

EleutherAI/wikitext_document_level

A modified version of the classic WikiText corpus, where texts are grouped into full pages.

What's inside

A text dataset with a volume of over 100 million tokens (from 10k to 100k records) extracted from verified Wikipedia featured and good articles. Unlike the original WikiText, which is split line by line, this version returns full articles at the document level. Distributed under a Creative Commons license.

What you can do with it

  • Training and evaluation of language models on long context (document-level)
  • Calculation of perplexity and classic text generation model benchmarking
Size
10K–100K rows
Downloads this month
106.1K
Likes
28

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("EleutherAI/wikitext_document_level")