Skip to content

Dataset · joelniklaus

LEXam hard legal exam questions

joelniklaus/LEXam-hard

A curated set of 518 challenging open-ended law exam questions from the University of Zurich for testing legal AI agents.

What's inside

A filtered test split of the LEXam benchmark (ICLR 2026) featuring 518 open-ended questions in English and German, ordered from hardest to easiest. Questions that strong models can solve from memory or that cannot be answered from the text alone were removed. Intended for evaluating systems capable of legal retrieval and analysis.

What you can do with it

  • Testing AI agents and RAG systems on legal analysis and retrieval tasks
  • Benchmarking LLM reasoning on complex European law
Tasks
Text generation, Question answering
Languages
English, German
Size
up to 1K rows
Downloads this month
739
Likes
32

How to load it

from datasets import load_dataset

ds = load_dataset("joelniklaus/LEXam-hard")