Dataset · common-pile
Large Stack Exchange Q&A corpus
common-pile/stackexchange
Tens of millions of English discussions from various Stack Exchange communities for text generation training.
What's inside
The dataset contains 10 to 100 million English text entries from thematic Q&A communities across the Stack Exchange network (including closed sections like windowsphone). Materials are gathered from site dumps of the platform, whose publication in the Internet Archive was discontinued in July 2024.
What you can do with it
- Pretraining and fine-tuning language models on technical and expert topics
- Prepare instruction data (SFT) in a question-and-detailed-answer format
- Populate knowledge bases for RAG systems on programming and related topics
- Tasks
- Text generation
- Languages
- English
- Size
- 10M–100M rows
- Downloads this month
- 9.2K
- Likes
- 11
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("common-pile/stackexchange")