Skip to content

Dataset · Open-Orca

Deduplicated instructions without RLHF

Open-Orca/SlimOrca-Dedup

363k unique instruction examples without RLHF, cleaned of duplicates for training generative models.

What's inside

This is a deduplicated and unfiltered subset of SlimOrca: 363k unique examples with RLHF examples removed. Duplicates were removed using minhash and Jaccard similarity. Demonstration models were trained on the full SlimOrca rather than this version.

What you can do with it

  • SFT fine-tuning of instruction LLMs
  • LoRA training for question-answering
  • Testing text generation on instructions
Tasks
Text classification, Question answering, Text generation
Size
100K–1M rows
Downloads this month
1.7K
Likes
96

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("Open-Orca/SlimOrca-Dedup")