Dataset · Skywork
Large Chinese web corpus for LLMs
Skywork/SkyPile-150B
Large filtered and deduplicated Chinese web text dataset for language model pre-training.
What's inside
SkyPile-150B is a large-scale Chinese text collection gathered from public web pages. The authors applied strict filtering, deduplication, and sensitive data removal, using fastText and BERT to weed out low-quality content. According to metadata, the size ranges from 1M to 10M records; the exact volume is not specified in the provided excerpt.
What you can do with it
- Pre-training Chinese LLMs
- Fine-tuning models on Chinese web texts
- Evaluating filtering and deduplication pipelines
- Tasks
- Text generation
- Languages
- Chinese
- Size
- 1M–10M rows
- Downloads this month
- 14.8K
- Likes
- 421
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("Skywork/SkyPile-150B")