Dataset · SakanaAI
Competitive programming benchmark from Sakana AI and AtCoder
SakanaAI/ALE-Bench
Official set of tasks from AtCoder contests for evaluating AI systems in algorithmic programming with scoring.
What's inside
The benchmark was developed by Sakana AI based on official materials from the Japanese platform AtCoder Inc. It contains less than a thousand tasks with texts and images in English and Japanese, designed to evaluate systems using a special Python toolkit. The set includes a research paper, GitHub code, and a leaderboard.
What you can do with it
- Evaluating the capabilities of LLMs and agents to solve complex optimization programming tasks
- Reinforcement learning research for generating and improving algorithmic code
- Tasks
- Image text → text, Reinforcement learning, Text generation, Visual question answering
- Languages
- English, Japanese
- Size
- up to 1K rows
- Downloads this month
- 20K
- Likes
- 13
How to load it
from datasets import load_dataset
ds = load_dataset("SakanaAI/ALE-Bench")