Dataset · RyokoAI
Articles from 23 thousand Fandom wiki projects
RyokoAI/Fandom23K
A massive English corpus of over 15.6 million articles from Fandom.com for pre-training and text generation.
What's inside
Contains 15,616,749 English text articles gathered from approximately 23,665 wikis on the Fandom.com platform in March 2023. The data is part of the larger BigKnow2022 corpus. The card notes warn that the dataset is not yet fully completed and some information may be inaccurate.
What you can do with it
- Unsupervised pre-training or continued pre-training (CPT) of language models on pop culture knowledge and lore
- Training text classification models and encyclopedic article generation
- Tasks
- Text classification, Text generation
- Languages
- English
- Size
- 1M–10M rows
- Downloads this month
- 882
- Likes
- 32
Models trained on it
- TheBloke/CausalLM-7B-GGUFText generation 6.6K
- TheBloke/CausalLM-14B-GGUFText generation 4.4K
- tastypear/CausalLM-7B-DPO-alpha-GGUFText generation 2.4K
- tastypear/CausalLM-14B-DPO-alpha-GGUFText generation 2.1K
- CausalLM/7B-DPO-alphaText generation 907
- mradermacher/35b-beta-long-i1-GGUF 766
- CausalLM/14BText generation 630
- bartowski/35b-beta-long-GGUFText generation 486
How to load it
from datasets import load_dataset
ds = load_dataset("RyokoAI/Fandom23K")