Dataset · nebius
Coding tasks benchmark in 13 rare languages
nebius/Longtail-SWE
471 real GitHub tasks for evaluating AI agents in writing software patches in less popular languages.
What's inside
Contains 471 real software development tasks in English based on GitHub issues and related pull requests. The dataset covers 13 programming languages: Clojure, Dart, Elixir, Erlang, F#, Haskell, Julia, Kotlin, Lua, OCaml, R, Scala, and Swift. The agent is given a problem description and a repository at a base commit, and the final patch is checked for passing tests.
What you can do with it
- Evaluating AI agents for programming in rare languages
- Comparing code generation models beyond Python and JS
- Testing automated bug fixing systems
- Tasks
- Text generation
- Languages
- English
- Size
- up to 1K rows
- Downloads this month
- 1.1K
- Likes
- 5
How to load it
from datasets import load_dataset
ds = load_dataset("nebius/Longtail-SWE")