Skip to content

Dataset · NousResearch

Benchmark on censorship and propaganda in LLM answers

NousResearch/Propaganda_Eval

803 prompts in English to test models for factualness and a bias toward official PRC propaganda.

What's inside

Inside are 803 text prompts in English on topics that are censored in the PRC or covered through an official agenda. Each question is accompanied by 2 to 4 mandatory facts from cited sources. Evaluation is based on two independent scales: factual accuracy of presentation (0-5) and reproduction of state propaganda (0-2).

What you can do with it

  • Evaluating the political neutrality and censorship of language models
  • Testing the factual accuracy of LLM answers on sensitive topics
  • Comparing the degree of ideological bias of different models
Tasks
Question answering
Languages
English
Size
up to 1K rows
Downloads this month
4.6K
Likes
78

How to load it

from datasets import load_dataset

ds = load_dataset("NousResearch/Propaganda_Eval")