Dataset · QuixiAI
Synthetic corpus for language model pretraining
QuixiAI/SYN-1B
Synthetic text streams with rules and links for LLM pretraining.
What's inside
The 1.04B token corpus in English consists of structured text streams where text establishes facts, mappings, and simple generative rules. Designed for structured pretraining rather than SFT.
What you can do with it
- Pretraining language models on structured synthetic rules
- Studying LLM capabilities in tracking logical connections and facts
- Tasks
- Text generation
- Languages
- English
- Size
- 1M–10M rows
- Downloads this month
- 509
- Likes
- 10
How to load it
from datasets import load_dataset
ds = load_dataset("QuixiAI/SYN-1B")