Dataset · common-pile
English public domain books from Project Gutenberg
common-pile/project_gutenberg
Tens of thousands of digitized English public domain books for pre-training generative models.
What's inside
The collection combines digitized English public domain book texts from the Project Gutenberg online library, along with books over 100 years old from the PG19 dataset. The corpus contains between 10,000 and 100,000 documents in total (the original library features over 75,000 books). Minimal preprocessing was applied to the texts to remove standard Project Gutenberg headers and footers.
What you can do with it
- Pre-training generative language models on long cohesive narrative and nonfiction texts
- Testing models on long-context processing
- Tasks
- Text generation
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 8.2K
- Likes
- 7
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("common-pile/project_gutenberg")