Dataset · monology
Uncopyrighted The Pile
monology/pile-uncopyrighted
A copy of The Pile corpus with removed books, subtitles, and other copyrighted texts.
What's inside
This is a copy of The Pile corpus from which copyrighted content has been removed to train LLMs without infringing authors' rights. According to the dataset card, the cleaning affected Books3, BookCorpus2, OpenSubtitles, YTSubtitles, and part of OWT2. The size ranges from 100 million to 1 billion items; the language is not specified in the metadata.
What you can do with it
- Pretraining LLMs without copyrighted data
- Comparing with the original The Pile
- Studying the impact of removing books and subtitles
- Size
- 100M–1B rows
- Downloads this month
- 60.5K
- Likes
- 193
Models trained on it
- MiniLLM/Ref-Pretrain-Qwen-104MText generation 909
- MiniLLM/Pretrain-Qwen-200MText generation 834
- MiniLLM/MiniPLM-Qwen-200MText generation 734
- mradermacher/MiniPLM-Qwen-500M-i1-GGUF 639
- mradermacher/MiniPLM-Qwen-500M-GGUF 444
- tensorblock/MiniPLM-Qwen-200M-GGUFText generation 415
- mradermacher/MiniPLM-Qwen-200M-i1-GGUF 369
- mradermacher/MiniPLM-Qwen-200M-GGUF 328
How to load it
from datasets import load_dataset
ds = load_dataset("monology/pile-uncopyrighted")