Skip to content

Dataset · futurehouse

BixBench data capsule benchmark for AI agents

futurehouse/BixBench

Updated benchmark with up to 1,000 tasks featuring the BixBench.jsonl questions file and data archives for model evaluation.

What's inside

The benchmark includes a questions file BixBench.jsonl and a set of associated "data capsules" in the form of zip archives (CapsuleFolder-{uuid}.zip). In September 2025, the dataset underwent a major revision: the authors reworked question formulations for accurate open-ended response evaluation and updated the file structure.

What you can do with it

  • Evaluating the ability of LLMs and AI agents to answer complex questions based on attached data archives
  • Testing analytical pipelines in an open-answer setting
Size
up to 1K rows
Downloads this month
43.6K
Likes
43

How to load it

from datasets import load_dataset

ds = load_dataset("futurehouse/BixBench")