Dataset · openbmb
Cleaned pre-training texts by UltraX
openbmb/UltraX-Preview
Large English corpus edited by the UltraX framework for language model pre-training.
What's inside
UltraX is a function-calling framework that adaptively generates and executes editing functions for point-wise cleaning of large-scale data. The dataset consists of English pre-training texts, with metadata indicating a size of 100M to 1B items. The dataset card links to the paper, code, and models, though the data format is unspecified.
What you can do with it
- Pre-training a small language model on a cleaned corpus
- Comparing training quality between cleaned and raw data
- Configuring a text-cleaning pipeline via function calling
- Tasks
- Text generation
- Languages
- English
- Size
- 100M–1B rows
- Downloads this month
- 6.6K
- Likes
- 302
Models trained on it
- openbmb/MiniCPM5-2BText generation 914.7K
- openbmb/MiniCPM5-2B-GGUFText generation 678.9K
- openbmb/MiniCPM5-2B-DSparkText generation 297K
- openbmb/MiniCPM5-2B-MLXText generation 137.4K
- openbmb/MiniCPM5-2B-BaseText generation 11K
- bartowski/MiniCPM5-2B-GGUFText generation 7.6K
- openbmb/MiniCPM5-2B-SFTText generation 6.6K
- openbmb/MiniCPM5-2B-MidtrainText generation 5.1K
How to load it
from datasets import load_dataset
ds = load_dataset("openbmb/UltraX-Preview")