Dataset · emily9589
PG-19 long context book corpus
emily9589/sn38-r18-2011-raw-pg19
A copy of the PG-19 benchmark based on Project Gutenberg books for training and evaluating language models on long texts.
What's inside
The repository contains the PG-19 text corpus (volume ranging from 100M to 1B entries) gathered from Project Gutenberg library books published before 1919, along with metadata on titles and publication dates. Documents in the set are on average 20 times longer than in the WikiText benchmark, and the total volume is more than double that of Billion Word.
What you can do with it
- Evaluating the ability of language models to maintain ultra-long context (long-range language modeling)
- Pre-training or fine-tuning text models on classical English literature
- Size
- 100M–1B rows
- Downloads this month
- 31.5K
- Likes
- 0
- Access
- on request on Hugging Face
How to load it
from datasets import load_dataset
ds = load_dataset("emily9589/sn38-r18-2011-raw-pg19")