Skip to content

Dataset · TAUR-Lab

Mystery-Based Multi-Step Reasoning Benchmark

TAUR-Lab/MuSR

English detective stories for testing LLM multi-step logical reasoning chains.

What's inside

An English text dataset (less than 1,000 examples) presented as a spotlight paper at ICLR 2024. It consists of ChatGPT-generated murder mystery riddles. Solving the tasks requires models to perform multi-step logical reasoning grounded in common sense.

What you can do with it

  • Testing Chain-of-Thought quality in language models
  • Evaluating LLM ability to maintain context and logic in complex scenarios
Tasks
Question answering
Languages
English
Size
up to 1K rows
Downloads this month
16.1K
Likes
28

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("TAUR-Lab/MuSR")