Skip to content

Dataset · IFM

Complete training corpus for open LLM Amber

IFM/AmberDatasets

Prepared English text data sequence used to train the open Amber model.

What's inside

Contains a fully prepared sequence of English texts ranging from 100M to 1B records. The data was collected and published as part of the LLM360 initiative, which makes the entire training process of large language models fully open. This is the official dataset used to train the Amber model.

What you can do with it

  • Replicating language model training from scratch using an open pipeline
  • Studying the impact of data sequencing order on LLM convergence
  • Pretraining custom English language models
Languages
English
Size
100M–1B rows
Downloads this month
30.7K
Likes
38

How to load it

from datasets import load_dataset

ds = load_dataset("IFM/AmberDatasets")