Skip to content

Dataset · SakanaAI

Competitive programming benchmark from Sakana AI and AtCoder

SakanaAI/ALE-Bench

Official set of tasks from AtCoder contests for evaluating AI systems in algorithmic programming with scoring.

What's inside

The benchmark was developed by Sakana AI based on official materials from the Japanese platform AtCoder Inc. It contains less than a thousand tasks with texts and images in English and Japanese, designed to evaluate systems using a special Python toolkit. The set includes a research paper, GitHub code, and a leaderboard.

What you can do with it

  • Evaluating the capabilities of LLMs and agents to solve complex optimization programming tasks
  • Reinforcement learning research for generating and improving algorithmic code
Tasks
Image text → text, Reinforcement learning, Text generation, Visual question answering
Languages
English, Japanese
Size
up to 1K rows
Downloads this month
20K
Likes
13

How to load it

from datasets import load_dataset

ds = load_dataset("SakanaAI/ALE-Bench")