Skip to content

Dataset · nebius

Coding tasks benchmark in 13 rare languages

nebius/Longtail-SWE

471 real GitHub tasks for evaluating AI agents in writing software patches in less popular languages.

What's inside

Contains 471 real software development tasks in English based on GitHub issues and related pull requests. The dataset covers 13 programming languages: Clojure, Dart, Elixir, Erlang, F#, Haskell, Julia, Kotlin, Lua, OCaml, R, Scala, and Swift. The agent is given a problem description and a repository at a base commit, and the final patch is checked for passing tests.

What you can do with it

  • Evaluating AI agents for programming in rare languages
  • Comparing code generation models beyond Python and JS
  • Testing automated bug fixing systems
Tasks
Text generation
Languages
English
Size
up to 1K rows
Downloads this month
1.1K
Likes
5

How to load it

from datasets import load_dataset

ds = load_dataset("nebius/Longtail-SWE")