Skip to content

Dataset · NeelNanda

First 10K texts from The Pile corpus for debugging

NeelNanda/pile-10k

A compact sample of the first 10,000 documents from the well-known The Pile text dataset for rapid testing.

What's inside

A small text dataset containing the first 10,000 items from the famous open corpus The Pile. Prepared by NeelNanda specifically for quick debugging and verification of models trained on the full The Pile, without needing to download the entire data volume.

What you can do with it

  • Rapid debugging of training pipelines and evaluation of language models
  • Checking perplexity and model behavior on baseline text distributions from The Pile
Size
10K–100K rows
Downloads this month
45.7K
Likes
37

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("NeelNanda/pile-10k")