Skip to content

Dataset · MiniMaxAI

Instruction-following benchmark for coding agents

MiniMaxAI/OctoCodingBench

An English benchmark checks whether a coding agent follows rules when working with a repository.

What's inside

OctoCodingBench is a text benchmark for agentic coding. It evaluates scaffold-aware instruction following in repository-bound tasks. The card notes that existing benchmarks like SWE-bench test task completion rather than rule adherence by the agent.

What you can do with it

  • Test a coding agent for rule compliance
  • Compare agent systems by instruction quality
  • Evaluate scaffold-aware model behavior
Tasks
Text generation
Languages
English
Size
up to 1K rows
Downloads this month
811
Likes
491

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("MiniMaxAI/OctoCodingBench")