Skip to content

Dataset · RyokoAI

Articles from 23 thousand Fandom wiki projects

RyokoAI/Fandom23K

A massive English corpus of over 15.6 million articles from Fandom.com for pre-training and text generation.

What's inside

Contains 15,616,749 English text articles gathered from approximately 23,665 wikis on the Fandom.com platform in March 2023. The data is part of the larger BigKnow2022 corpus. The card notes warn that the dataset is not yet fully completed and some information may be inaccurate.

What you can do with it

  • Unsupervised pre-training or continued pre-training (CPT) of language models on pop culture knowledge and lore
  • Training text classification models and encyclopedic article generation
Tasks
Text classification, Text generation
Languages
English
Size
1M–10M rows
Downloads this month
882
Likes
32

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("RyokoAI/Fandom23K")