Dataset · lmsys
ToxicChat: toxic prompts
lmsys/toxic-chat
10K prompts from Vicuna demo with toxicity annotations for classification.
What's inside
ToxicChat contains toxicity annotations for 10K user prompts collected from online Vicuna demos. There is the toxicchat0124 version, updated OpenAI Moderation API results, and the official T5-Large baseline. The language is English, and modalities are text and tabular.
What you can do with it
- Training a toxicity classifier
- Evaluating chatbot moderation
- Comparing with the T5-Large baseline
- Tasks
- Text classification
- Languages
- English
- Size
- 10K–100K rows
- Downloads this month
- 9.8K
- Likes
- 215
Models trained on it
- vllm-sr/mmbert32k-jailbreak-detector-mergedText classification 2.8K
- nickprock/archai-jev-qwen-1.5bText generation 636
- OpenSafetyLab/MD-Judge-v0.1Text generation 526
- DominicTWHV/Horizon-1-Text-LargeText classification 151
- KucLab/Censer.1Text classification 114
- Verm1ion/injection-sentry-xlmrText classification 103
- KucLab/Barist.1Text classification 101
- opus-research/opus-moderation-2Text classification 86
How to load it
from datasets import load_dataset
ds = load_dataset("lmsys/toxic-chat")