Dataset · NousResearch
Benchmark on censorship and propaganda in LLM answers
NousResearch/Propaganda_Eval
803 prompts in English to test models for factualness and a bias toward official PRC propaganda.
What's inside
Inside are 803 text prompts in English on topics that are censored in the PRC or covered through an official agenda. Each question is accompanied by 2 to 4 mandatory facts from cited sources. Evaluation is based on two independent scales: factual accuracy of presentation (0-5) and reproduction of state propaganda (0-2).
What you can do with it
- Evaluating the political neutrality and censorship of language models
- Testing the factual accuracy of LLM answers on sensitive topics
- Comparing the degree of ideological bias of different models
- Tasks
- Question answering
- Languages
- English
- Size
- up to 1K rows
- Downloads this month
- 4.6K
- Likes
- 78
How to load it
from datasets import load_dataset
ds = load_dataset("NousResearch/Propaganda_Eval")