Skip to content

Dataset · IFM

TxT360-v2: web and QA texts

IFM/TxT360-v2

Corpus of web and question-answering texts for model pre-training from the K2 Horizon release.

What's inside

This is part of the K2 Horizon collection of raw pre-training data. The repository is split into several subsets; this one contains web texts and question-answering texts across 3 subsets. Each subset has a train split consisting of parquet shards, with a metadata volume of 1 to 10 billion.

What you can do with it

  • Pre-train LLMs on web and QA texts
  • Continue pre-training on K2 Horizon parquet shards
  • Test the K2 Horizon data loading pipeline
Tasks
Text generation
Size
1B–10B rows
Downloads this month
74.9K
Likes
108

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("IFM/TxT360-v2")