Dataset · HuggingFaceGECLM
Archive of posts from 50 popular subreddits
HuggingFaceGECLM/REDDIT_submissions
Tens of millions of Reddit posts from 2006–2023, split by thematic communities for training language models.
What's inside
Massive English text corpus with 10 to 100 million records, extracted from PushShift dumps spanning from 2006 to January 2023. The data includes publications from 50 selected subreddits such as tifu and explainlikeimfive. Each split in the dataset corresponds to an individual community.
What you can do with it
- Pre-training and fine-tuning language models on live conversational speech
- Training dialogue systems and text generators for specific topics or styles
- Tasks
- Text generation
- Languages
- English
- Size
- 10M–100M rows
- Downloads this month
- 3.2K
- Likes
- 15
How to load it
from datasets import load_dataset
ds = load_dataset("HuggingFaceGECLM/REDDIT_submissions")