Skip to content

Dataset · mmathys

OpenAI test set for unwanted content moderation

mmathys/openai-moderation-api-evaluation

Reference dataset from the OpenAI paper with text labeled by violation categories to evaluate moderation systems.

What's inside

Contains English text and tabular data packaged in a JSONL archive (samples-1680.jsonl.gz) with a total volume of 1,000 to 10,000 examples. Each object is labeled with binary flags according to undesired content taxonomy (e.g., sexual content and other categories). This is a test set from the OpenAI research paper "A Holistic Approach to Undesired Content Detection".

What you can do with it

  • Benchmarking custom text safety and moderation filters
  • Evaluating toxicity and restricted content detection quality in LLM pipelines
  • Comparing commercial and open moderation APIs on a benchmark test
Tasks
Text classification
Languages
English
Size
1K–10K rows
Downloads this month
1.4K
Likes
39

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("mmathys/openai-moderation-api-evaluation")