Dataset · HuggingFaceFW
FineWeb-2 multilingual corpus
HuggingFaceFW/fineweb-2
FineWeb-2 pretraining corpus with 1000+ languages for training language models.
What's inside
The second release of FineWeb: high-quality texts for pretraining across more than 1000 languages. The dataset is fully reproducible, available under the ODC-By 1.0 license, and verified by hundreds of ablation experiments; the authors adjusted processing for 9 languages. Format is tabular/text, size ranges from 1B to 10B examples.
What you can do with it
- Pretraining multilingual LLMs
- Continued pretraining for low-resource languages
- Evaluating the quality of multilingual corpora
- Tasks
- Text generation
- Languages
- aai, aak, aau, aaz, aba, abi, Abkhazian, abn and 4 more
- Size
- 1B–10B rows
- Downloads this month
- 205.9K
- Likes
- 939
How to load it
from datasets import load_dataset
ds = load_dataset("HuggingFaceFW/fineweb-2")