Skip to content

Dataset · SciCode1

SciCode scientific computing and programming benchmark

SciCode1/SciCode

Scientific coding task dataset authored by researchers to evaluate LLMs.

What's inside

An English-language text benchmark in a question-and-answer format, presented in the SciCode paper. The tasks were authored by scientists to evaluate the ability of language models to write code for scientific research. The dataset size is less than a thousand examples.

What you can do with it

  • Evaluating LLM ability to generate correct code for scientific tasks
  • Comparing reasoning and coding models on complex research scenarios
Tasks
Question answering
Languages
English
Size
up to 1K rows
Downloads this month
40.8K
Likes
19

How to load it

from datasets import load_dataset

ds = load_dataset("SciCode1/SciCode")