Skip to content

Dataset · zai-org

Long instructions for LLM alignment

zai-org/LongAlign-10k

10k long instructions of 8–64K tokens in en/zh for training and evaluating long-context LLMs.

What's inside

The LongAlign-10k dataset contains 10,000 long instructions ranging from 8k to 64k tokens in English and Chinese. It is part of the LongAlign recipe for long-context LLM alignment; the card also mentions packing and sorted batching strategies and the LongBench-Chat benchmark. The format is not specified in the metadata.

What you can do with it

  • SFT for long-context LLMs
  • Evaluation on LongBench-Chat
  • LoRA for long instructions
Tasks
Question answering
Languages
English, Chinese
Size
1K–10K rows
Downloads this month
4.2K
Likes
100

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("zai-org/LongAlign-10k")