Skip to content

Dataset · storytracer

Digitized books from the US Library of Congress

storytracer/LoC-PD-Books

Massive corpus of over 140,000 public domain English books with OCR texts.

What's inside

The dataset contains over 140,000 digitized English books (about 8 billion words) in the public domain in the US. The materials were collected via the Library of Congress JSON API by filtering the Selected Digitized Books collection and contain recognized OCR texts. The metadata specifies text and tabular modalities, and the file format is not specified.

What you can do with it

  • Pre-training or continued training of large language models on long texts
  • Training models to handle historical English and literary style
  • Research in digital humanities and OCR error post-processing
Tasks
Text generation
Languages
English
Size
10K–100K rows
Downloads this month
1.6K
Likes
44

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("storytracer/LoC-PD-Books")