Dataset · m-a-p
800-billion-token Chinese pre-training corpus
m-a-p/MAP-CC
Open Chinese pre-training dataset of 800 billion tokens for the NLP community.
What's inside
This is an open Chinese pre-training corpus on a scale of 800 billion tokens. According to the card, the data passed compliance checks. The modalities are image and text, and the metadata size is from 1 to 10 billion records.
What you can do with it
- Pre-training a Chinese language model
- Continuing pre-training of a multimodal model
- Evaluating the quality of Chinese corpora
- Size
- 1B–10B rows
- Downloads this month
- 9.7K
- Likes
- 83
How to load it
from datasets import load_dataset
ds = load_dataset("m-a-p/MAP-CC")