Dataset · IFM
Complete training corpus for open LLM Amber
IFM/AmberDatasets
Prepared English text data sequence used to train the open Amber model.
What's inside
Contains a fully prepared sequence of English texts ranging from 100M to 1B records. The data was collected and published as part of the LLM360 initiative, which makes the entire training process of large language models fully open. This is the official dataset used to train the Amber model.
What you can do with it
- Replicating language model training from scratch using an open pipeline
- Studying the impact of data sequencing order on LLM convergence
- Pretraining custom English language models
- Languages
- English
- Size
- 100M–1B rows
- Downloads this month
- 30.7K
- Likes
- 38
How to load it
from datasets import load_dataset
ds = load_dataset("IFM/AmberDatasets")