Dataset · MC7ever
Multimodal HF Corpus of 92K Datasets
MC7ever/hf-training-corpus
Large-scale extraction of texts, images, audio, and video from open datasets in a unified parquet format.
What's inside
A meta-dataset with a volume of 1 to 10 million English-language records, collected by automatically crawling about 92,000 datasets from Hugging Face. Different modalities are normalized and packed into parquet files: texts, images (PNG), audio (WAV), video fragments, and tabular data converted to text.
What you can do with it
- Pretraining multimodal foundational models on heterogeneous data
- Creating universal pipelines for training text generation, ASR, and image analysis models
- Tasks
- Text generation, Image → text, Automatic speech recognition
- Languages
- English
- Size
- 100K–1M rows
- Downloads this month
- 26.8K
- Likes
- 1
How to load it
from datasets import load_dataset
ds = load_dataset("MC7ever/hf-training-corpus")