Skip to content

Dataset · HuggingFaceTB

Cosmopedia large English text corpus

HuggingFaceTB/cosmopedia

Large English text corpus for training and fine-tuning language models.

What's inside

Cosmopedia v0.1 is a large English text corpus with a volume of 10 to 100 million examples. The card notes that illustration prompts were generated by Mixtral-8x7B-Instruct-v0.1 and images by DALL-E. Version v0.2 is available in smollm-corpus; format is text.

What you can do with it

  • Pre-training or continued pre-training of a small language model
  • LoRA fine-tuning for English text generation
  • Testing the preparation pipeline for large text corpora
Languages
English
Size
10M–100M rows
Downloads this month
27.8K
Likes
777

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("HuggingFaceTB/cosmopedia")