Dataset · nvidia
NVIDIA LLM throughput benchmark
nvidia/SPEED-Bench
Long-prompt collection (up to 100K tokens) with varying entropy for measuring language model inference speed and quality.
What's inside
Repository from NVIDIA containing 1k to 10k text and document examples, updated to version v1.1 in September 2026. Includes splits for evaluating throughput at 16K, 32K, and 100K context lengths, categorized by entropy level (low, mixed, high) and including code debugging tasks from InfiniteBench and RepoBench. Prompts are structured with the instruction placed at the very beginning before the long context.
What you can do with it
- Measure LLM generation throughput and latency on long contexts.
- Test speculative decoding methods on tasks with varying entropy.
- Compare inference engines on contexts up to 100K tokens.
- Size
- 1K–10K rows
- Downloads this month
- 27.1K
- Likes
- 54
How to load it
from datasets import load_dataset
ds = load_dataset("nvidia/SPEED-Bench")