Skip to content

Dataset · instruction-pretrain

Data for training context-based instruction generators

instruction-pretrain/ft-instruction-synthesizer-collection

English corpus (100K–1M examples) accompanying the EMNLP 2024 paper for training an instruction and response synthesizer.

What's inside

An English text dataset ranging from 100,000 to 1,000,000 records, created as part of the Instruction Pre-Training (EMNLP 2024) study. It is designed for fine-tuning a synthesizer model that turns raw texts into instruction-response pairs for multi-task pre-training.

What you can do with it

  • Training models to generate synthetic instructions from unlabeled texts
  • Preparing data for multi-task pre-training and SFT in classification and QA tasks
Tasks
Text classification, Table question answering, Question answering, Zero shot classification
Languages
English
Size
100K–1M rows
Downloads this month
297
Likes
63

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("instruction-pretrain/ft-instruction-synthesizer-collection")