Skip to content

Dataset · HuggingFaceGECLM

Archive of posts from 50 popular subreddits

HuggingFaceGECLM/REDDIT_submissions

Tens of millions of Reddit posts from 2006–2023, split by thematic communities for training language models.

What's inside

Massive English text corpus with 10 to 100 million records, extracted from PushShift dumps spanning from 2006 to January 2023. The data includes publications from 50 selected subreddits such as tifu and explainlikeimfive. Each split in the dataset corresponds to an individual community.

What you can do with it

  • Pre-training and fine-tuning language models on live conversational speech
  • Training dialogue systems and text generators for specific topics or styles
Tasks
Text generation
Languages
English
Size
10M–100M rows
Downloads this month
3.2K
Likes
15

How to load it

from datasets import load_dataset

ds = load_dataset("HuggingFaceGECLM/REDDIT_submissions")