Dataset · tabularisai
650M+ token synthetic corpus for LLM training
tabularisai/oak
A large-scale set of English synthetic texts generated by an ensemble of language models.
What's inside
The dataset includes between 1M and 10M English records totaling over 650 million tokens (accounting for the v1.1 update of 120M tokens). The data is presented in tabular and text formats. The corpus is fully generated using an ensemble of modern LLMs as an ethical and diverse data source for training.
What you can do with it
- Pre-training and fine-tuning large language models (LLMs)
- Knowledge distillation from an ensemble of powerful models into more compact ones
- Tasks
- Text generation
- Languages
- English
- Size
- 1M–10M rows
- Downloads this month
- 770
- Likes
- 38
How to load it
from datasets import load_dataset
ds = load_dataset("tabularisai/oak")