Skip to content

Dataset · HuggingFaceFW

FineWeb-2 multilingual corpus

HuggingFaceFW/fineweb-2

FineWeb-2 pretraining corpus with 1000+ languages for training language models.

What's inside

The second release of FineWeb: high-quality texts for pretraining across more than 1000 languages. The dataset is fully reproducible, available under the ODC-By 1.0 license, and verified by hundreds of ablation experiments; the authors adjusted processing for 9 languages. Format is tabular/text, size ranges from 1B to 10B examples.

What you can do with it

  • Pretraining multilingual LLMs
  • Continued pretraining for low-resource languages
  • Evaluating the quality of multilingual corpora
Tasks
Text generation
Languages
aai, aak, aau, aaz, aba, abi, Abkhazian, abn and 4 more
Size
1B–10B rows
Downloads this month
205.9K
Likes
939

How to load it

from datasets import load_dataset

ds = load_dataset("HuggingFaceFW/fineweb-2")