Dataset · storytracer
German historical newspaper archive (13B words)
storytracer/German-PD-Newspapers
Massive corpus of OCR texts from public domain old German newspapers for training language models.
What's inside
Contains 13 billion words (1 to 10 million records) in German, obtained via OCR from historical newspapers. Data is sourced from the Deutsche Digitale Bibliothek repository and distributed under free CC0 and Public Domain licenses.
What you can do with it
- Pre-train or fine-tune German-language LLMs from scratch
- Adapt language models to historical German spelling and vocabulary
- Train OCR post-processing systems for archival documents
- Tasks
- Text generation
- Languages
- German
- Size
- 1M–10M rows
- Downloads this month
- 820
- Likes
- 8
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("storytracer/German-PD-Newspapers")