Dataset · joelniklaus
LEXam hard legal exam questions
joelniklaus/LEXam-hard
A curated set of 518 challenging open-ended law exam questions from the University of Zurich for testing legal AI agents.
What's inside
A filtered test split of the LEXam benchmark (ICLR 2026) featuring 518 open-ended questions in English and German, ordered from hardest to easiest. Questions that strong models can solve from memory or that cannot be answered from the text alone were removed. Intended for evaluating systems capable of legal retrieval and analysis.
What you can do with it
- Testing AI agents and RAG systems on legal analysis and retrieval tasks
- Benchmarking LLM reasoning on complex European law
- Tasks
- Text generation, Question answering
- Languages
- English, German
- Size
- up to 1K rows
- Downloads this month
- 739
- Likes
- 32
How to load it
from datasets import load_dataset
ds = load_dataset("joelniklaus/LEXam-hard")