Dataset · IFM
TxT360-v2: web and QA texts
IFM/TxT360-v2
Corpus of web and question-answering texts for model pre-training from the K2 Horizon release.
What's inside
This is part of the K2 Horizon collection of raw pre-training data. The repository is split into several subsets; this one contains web texts and question-answering texts across 3 subsets. Each subset has a train split consisting of parquet shards, with a metadata volume of 1 to 10 billion.
What you can do with it
- Pre-train LLMs on web and QA texts
- Continue pre-training on K2 Horizon parquet shards
- Test the K2 Horizon data loading pipeline
- Tasks
- Text generation
- Size
- 1B–10B rows
- Downloads this month
- 74.9K
- Likes
- 108
Models trained on it
- MinimaLabs/KLM-1.6Text generation 3.2K
- AxiomicLabs/GPT-X3-150MText generation 2.1K
- MinimaLabs/KLM-1.5Text generation 1.5K
- techdotus/Ethoric-10MText generation 1.3K
- kash8080/Mantis-50M-BaseText generation 1K
- kash8080/Mantis-50MText generation 839
- MinimaLabs/min-spark-1.7Text generation 723
- AxiomicLabs/GPT-S3-3MText generation 720
How to load it
from datasets import load_dataset
ds = load_dataset("IFM/TxT360-v2")