Skip to content

Dataset · monology

Uncopyrighted The Pile

monology/pile-uncopyrighted

A copy of The Pile corpus with removed books, subtitles, and other copyrighted texts.

What's inside

This is a copy of The Pile corpus from which copyrighted content has been removed to train LLMs without infringing authors' rights. According to the dataset card, the cleaning affected Books3, BookCorpus2, OpenSubtitles, YTSubtitles, and part of OWT2. The size ranges from 100 million to 1 billion items; the language is not specified in the metadata.

What you can do with it

  • Pretraining LLMs without copyrighted data
  • Comparing with the original The Pile
  • Studying the impact of removing books and subtitles
Size
100M–1B rows
Downloads this month
60.5K
Likes
193

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("monology/pile-uncopyrighted")