Skip to content

Dataset · coral-nlp

154.56B Tokens of Open German Text

coral-nlp/german-commons

A large corpus of openly licensed German texts from 41 sources for training language models.

What's inside

Combines German-language texts and tabular data from 41 different sources under open licenses. The total volume is 154.56 billion tokens (10 million to 100 million records), with an included Bloom filter compatible with DOLMA. The corpus is designed for training German text generation models.

What you can do with it

  • Pretraining and fine-tuning German LLMs
  • Filtering and deduplication of corpora using the included bloom_filter.bin
  • Research in German text generation
Tasks
Text generation
Languages
German
Size
10M–100M rows
Downloads this month
3.6K
Likes
42

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("coral-nlp/german-commons")