Dataset · instruction-pretrain
Data for training context-based instruction generators
instruction-pretrain/ft-instruction-synthesizer-collection
English corpus (100K–1M examples) accompanying the EMNLP 2024 paper for training an instruction and response synthesizer.
What's inside
An English text dataset ranging from 100,000 to 1,000,000 records, created as part of the Instruction Pre-Training (EMNLP 2024) study. It is designed for fine-tuning a synthesizer model that turns raw texts into instruction-response pairs for multi-task pre-training.
What you can do with it
- Training models to generate synthetic instructions from unlabeled texts
- Preparing data for multi-task pre-training and SFT in classification and QA tasks
- Tasks
- Text classification, Table question answering, Question answering, Zero shot classification
- Languages
- English
- Size
- 100K–1M rows
- Downloads this month
- 297
- Likes
- 63
Models trained on it
- QuantFactory/InstructLM-500M-GGUFText generation 728
- QuantFactory/instruction-synthesizer-GGUFText generation 668
- QuantFactory/InstructLM-1.3B-GGUFText generation 589
- mradermacher/InstructLM-500M-i1-GGUF 587
- mradermacher/InstructLM-1.3B-i1-GGUF 379
- mradermacher/instruction-synthesizer-i1-GGUF 358
- instruction-pretrain/InstructLM-500MText generation 280
- instruction-pretrain/InstructLM-1.3BText generation 251
How to load it
from datasets import load_dataset
ds = load_dataset("instruction-pretrain/ft-instruction-synthesizer-collection")