Dataset · TAUR-Lab
Mystery-Based Multi-Step Reasoning Benchmark
TAUR-Lab/MuSR
English detective stories for testing LLM multi-step logical reasoning chains.
What's inside
An English text dataset (less than 1,000 examples) presented as a spotlight paper at ICLR 2024. It consists of ChatGPT-generated murder mystery riddles. Solving the tasks requires models to perform multi-step logical reasoning grounded in common sense.
What you can do with it
- Testing Chain-of-Thought quality in language models
- Evaluating LLM ability to maintain context and logic in complex scenarios
- Tasks
- Question answering
- Languages
- English
- Size
- up to 1K rows
- Downloads this month
- 16.1K
- Likes
- 28
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("TAUR-Lab/MuSR")