Dataset · zai-org
Long instructions for LLM alignment
zai-org/LongAlign-10k
10k long instructions of 8–64K tokens in en/zh for training and evaluating long-context LLMs.
What's inside
The LongAlign-10k dataset contains 10,000 long instructions ranging from 8k to 64k tokens in English and Chinese. It is part of the LongAlign recipe for long-context LLM alignment; the card also mentions packing and sorted batching strategies and the LongBench-Chat benchmark. The format is not specified in the metadata.
What you can do with it
- SFT for long-context LLMs
- Evaluation on LongBench-Chat
- LoRA for long instructions
- Tasks
- Question answering
- Languages
- English, Chinese
- Size
- 1K–10K rows
- Downloads this month
- 4.2K
- Likes
- 100
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("zai-org/LongAlign-10k")