Skip to content

Dataset · QuixiAI

Synthetic corpus for language model pretraining

QuixiAI/SYN-1B

Synthetic text streams with rules and links for LLM pretraining.

What's inside

The 1.04B token corpus in English consists of structured text streams where text establishes facts, mappings, and simple generative rules. Designed for structured pretraining rather than SFT.

What you can do with it

  • Pretraining language models on structured synthetic rules
  • Studying LLM capabilities in tracking logical connections and facts
Tasks
Text generation
Languages
English
Size
1M–10M rows
Downloads this month
509
Likes
10

How to load it

from datasets import load_dataset

ds = load_dataset("QuixiAI/SYN-1B")