Dataset · MiniMaxAI
Instruction-following benchmark for coding agents
MiniMaxAI/OctoCodingBench
An English benchmark checks whether a coding agent follows rules when working with a repository.
What's inside
OctoCodingBench is a text benchmark for agentic coding. It evaluates scaffold-aware instruction following in repository-bound tasks. The card notes that existing benchmarks like SWE-bench test task completion rather than rule adherence by the agent.
What you can do with it
- Test a coding agent for rule compliance
- Compare agent systems by instruction quality
- Evaluate scaffold-aware model behavior
- Tasks
- Text generation
- Languages
- English
- Size
- up to 1K rows
- Downloads this month
- 811
- Likes
- 491
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("MiniMaxAI/OctoCodingBench")