Skip to content

Dataset · LiquidAI

JSON/YAML structured output benchmark

LiquidAI/ifstruct-v1.0

1K-10K examples to test if a model follows a given JSON/YAML schema under various prompt formulations.

What's inside

IFStruct v1.0 is a structured output compliance benchmark: the model must output valid JSON/YAML according to a requested schema. Requirements are phrased as real users write them. Evaluation runs without constrained decoding and checks only structure, not content quality, extraction, or reasoning. Language is English, size is 1K to 10K examples.

What you can do with it

  • Evaluating models on structured output compliance
  • Checking JSON/YAML schemas without constrained decoding
  • Comparing prompts for valid JSON generation
Tasks
Text generation
Languages
English
Size
1K–10K rows
Downloads this month
1.1K
Likes
114

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("LiquidAI/ifstruct-v1.0")