Skip to content

Dataset · tabularisai

650M+ token synthetic corpus for LLM training

tabularisai/oak

A large-scale set of English synthetic texts generated by an ensemble of language models.

What's inside

The dataset includes between 1M and 10M English records totaling over 650 million tokens (accounting for the v1.1 update of 120M tokens). The data is presented in tabular and text formats. The corpus is fully generated using an ensemble of modern LLMs as an ethical and diverse data source for training.

What you can do with it

  • Pre-training and fine-tuning large language models (LLMs)
  • Knowledge distillation from an ensemble of powerful models into more compact ones
Tasks
Text generation
Languages
English
Size
1M–10M rows
Downloads this month
770
Likes
38

How to load it

from datasets import load_dataset

ds = load_dataset("tabularisai/oak")