Skip to content

Dataset · allenai

C4 cleaned web corpus

allenai/c4

Massive cleaned Common Crawl corpus with en, realnewslike, and multilingual mC4 variants.

What's inside

C4 is a cleaned version of the Common Crawl web crawl, based on Google's processed C4. Available variants include en, en.noclean, en.noblocklist, realnewslike, and multilingual mC4. Variant sizes: en - 305 GB, en.noclean - 2.3 TB, en.noblocklist - 380 GB, realnewslike - 15 GB, mC4 - 9.7 TB across 108 subsets.

What you can do with it

  • Pre-training language models
  • Fine-tuning on English web text
  • Experiments with realnewslike for the news domain
Tasks
Text generation, Fill mask
Languages
Afrikaans, Amharic, Arabic, Azerbaijani, Belarusian, Bulgarian, Bangla, Catalan and 4 more
Size
10B–100B rows
Downloads this month
775.4K
Likes
705

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("allenai/c4")