Dataset · HuggingFaceTB
Cosmopedia large English text corpus
HuggingFaceTB/cosmopedia
Large English text corpus for training and fine-tuning language models.
What's inside
Cosmopedia v0.1 is a large English text corpus with a volume of 10 to 100 million examples. The card notes that illustration prompts were generated by Mixtral-8x7B-Instruct-v0.1 and images by DALL-E. Version v0.2 is available in smollm-corpus; format is text.
What you can do with it
- Pre-training or continued pre-training of a small language model
- LoRA fine-tuning for English text generation
- Testing the preparation pipeline for large text corpora
- Languages
- English
- Size
- 10M–100M rows
- Downloads this month
- 27.8K
- Likes
- 777
Models trained on it
- mradermacher/Mistral-7B-Instruct-KhanAcademy-v0.2-i1-GGUF 3.7K
- MinimaLabs/KLM-1.6Text generation 3.2K
- DeepMount00/Alireo-400m-instruct-v0.1Text generation 2.3K
- mradermacher/SpydazWeb_AI_LCARS_Humanization_003-i1-GGUF 1.9K
- MinimaLabs/KLM-1.5Text generation 1.5K
- nkthebass/tinybrainbot-3m-loopText generation 1.5K
- nkthebass/tinybrainbot-3m-v2-baseText generation 1.5K
- nkthebass/tinybrainbot-100m-v5.1-baseText generation 1.5K
How to load it
from datasets import load_dataset
ds = load_dataset("HuggingFaceTB/cosmopedia")