Skip to content

Dataset · common-pile

English public domain books from Project Gutenberg

common-pile/project_gutenberg

Tens of thousands of digitized English public domain books for pre-training generative models.

What's inside

The collection combines digitized English public domain book texts from the Project Gutenberg online library, along with books over 100 years old from the PG19 dataset. The corpus contains between 10,000 and 100,000 documents in total (the original library features over 75,000 books). Minimal preprocessing was applied to the texts to remove standard Project Gutenberg headers and footers.

What you can do with it

  • Pre-training generative language models on long cohesive narrative and nonfiction texts
  • Testing models on long-context processing
Tasks
Text generation
Languages
English
Size
10K–100K rows
Downloads this month
8.2K
Likes
7

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("common-pile/project_gutenberg")