Skip to content

Dataset · nvidia

NVIDIA LLM throughput benchmark

nvidia/SPEED-Bench

Long-prompt collection (up to 100K tokens) with varying entropy for measuring language model inference speed and quality.

What's inside

Repository from NVIDIA containing 1k to 10k text and document examples, updated to version v1.1 in September 2026. Includes splits for evaluating throughput at 16K, 32K, and 100K context lengths, categorized by entropy level (low, mixed, high) and including code debugging tasks from InfiniteBench and RepoBench. Prompts are structured with the instruction placed at the very beginning before the long context.

What you can do with it

  • Measure LLM generation throughput and latency on long contexts.
  • Test speculative decoding methods on tasks with varying entropy.
  • Compare inference engines on contexts up to 100K tokens.
Size
1K–10K rows
Downloads this month
27.1K
Likes
54

How to load it

from datasets import load_dataset

ds = load_dataset("nvidia/SPEED-Bench")