Skip to content

Dataset · 0xDing

Filtered Chinese Wikipedia 2023

0xDing/wikipedia-cn-20230720-filtered

254k cleaned Chinese Wikipedia articles for training generative models in Chinese.

What's inside

These are Chinese Wikipedia article texts from the July 20, 2023 dump. 254,547 high-quality articles are retained; system pages, low-quality parts, and sensitive or controversial materials are filtered out. Simplification and word replacement with Mainland China standards were performed.

What you can do with it

  • Fine-tune a Chinese language model on clean articles
  • Train LoRA for text generation in Chinese
  • Verify generation quality on wiki texts
Tasks
Text generation
Languages
Chinese
Size
100K–1M rows
Downloads this month
4.9K
Likes
182

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("0xDing/wikipedia-cn-20230720-filtered")