Skip to content

Dataset · akshathmangudi

SciCode scientific code generation benchmark

akshathmangudi/SciCode

Complex English text tasks for evaluating the ability of language models to write code for scientific research.

What's inside

The dataset contains between 1,000 and 10,000 text tasks created with the participation of scientists and AI researchers from 16 different natural science fields. It is designed to realistically test how well language models handle code generation for solving real research problems.

What you can do with it

  • Evaluating and comparing language models in scientific programming tasks
  • Testing the ability of LLMs to solve complex applied problems from natural sciences
Tasks
Question answering
Languages
English
Size
1K–10K rows
Downloads this month
20.4K
Likes
0

How to load it

from datasets import load_dataset

ds = load_dataset("akshathmangudi/SciCode")