Dataset · storytracer
Digitized books from the US Library of Congress
storytracer/LoC-PD-Books
Massive corpus of over 140,000 public domain English books with OCR texts.
What's inside
The dataset contains over 140,000 digitized English books (about 8 billion words) in the public domain in the US. The materials were collected via the Library of Congress JSON API by filtering the Selected Digitized Books collection and contain recognized OCR texts. The metadata specifies text and tabular modalities, and the file format is not specified.
What you can do with it
- Pre-training or continued training of large language models on long texts
- Training models to handle historical English and literary style
- Research in digital humanities and OCR error post-processing
- Tasks
- Text generation
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 1.6K
- Likes
- 44
Models trained on it
- bartowski/YuLan-Mini-GGUFText generation 1.4K
- diskrot/YuLan-Mini-diskrotText generation 630
- ikarth/openpd-gpt2-v0Text generation 618
- tensorblock/YuLan-Mini-GGUFText generation 365
- mradermacher/YuLan-Mini-GGUF 362
- diskrot/YuLan-Mini-GGUF-diskrotText generation 352
- QuantFactory/YuLan-Mini-GGUFText generation 299
- ikarth/openpd-gpt2-v1Text generation 121
How to load it
from datasets import load_dataset
ds = load_dataset("storytracer/LoC-PD-Books")