Dataset · ksolovev
Cleaned corpus of 635M news articles in six languages
ksolovev/fine-news
Large-scale multilingual archive of news articles from 2016–2026 with quality filtering and deduplication.
What's inside
The corpus contains 635,068,343 cleaned news documents in English, Spanish, Russian, Italian, German, and French in text and tabular format. Articles were extracted from Common Crawl News web archives for the period from August 2016 to April 2026 via the INFINI-NEWS project. Each document underwent language verification, quality filtering, duplicate removal, and received a sampling weight.
What you can do with it
- Pre-training and fine-tuning of language models, including Russian-language models
- Training of retrieval and news text classification systems
- Tasks
- Text generation, Text classification, Text retrieval
- Languages
- English, Spanish, Russian, Italian, German, French, Turkish, Arabic and 4 more
- Size
- 100M–1B rows
- Downloads this month
- 459.8K
- Likes
- 53
How to load it
from datasets import load_dataset
ds = load_dataset("ksolovev/fine-news")