Skip to content

Dataset · ksolovev

Cleaned corpus of 635M news articles in six languages

ksolovev/fine-news

Large-scale multilingual archive of news articles from 2016–2026 with quality filtering and deduplication.

What's inside

The corpus contains 635,068,343 cleaned news documents in English, Spanish, Russian, Italian, German, and French in text and tabular format. Articles were extracted from Common Crawl News web archives for the period from August 2016 to April 2026 via the INFINI-NEWS project. Each document underwent language verification, quality filtering, duplicate removal, and received a sampling weight.

What you can do with it

  • Pre-training and fine-tuning of language models, including Russian-language models
  • Training of retrieval and news text classification systems
Tasks
Text generation, Text classification, Text retrieval
Languages
English, Spanish, Russian, Italian, German, French, Turkish, Arabic and 4 more
Size
100M–1B rows
Downloads this month
459.8K
Likes
53

How to load it

from datasets import load_dataset

ds = load_dataset("ksolovev/fine-news")