Skip to content

Datasets

Hugging Face datasets with plain-language descriptions: what's inside and what models to train on them. 4,788 in the catalog so far, with new ones added every hour.

Data type

For which models

Find a dataset for your goal

Answered by Gemini 4 · the chat is kept only in this tab

Datasetkurumikz/AnimeOmni

Anime characters with multi-level annotation and descriptions

Over 20 thousand safe-rated anime illustrations with detailed tags, emotions, and descriptions in three languages.

  • Text → image
  • Image → text
  • Image classification
  • English
  • Chinese
  • Japanese
5.5K 1310K–100K rows
Datasetvidore/colpali_train_set

ColPali training set for document retrieval

127K query-image pairs for training visual document retrieval.

  • Document question answering
  • Visual document retrieval
5.3K 93100K–1M rows
DatasetTHUIR/Qilin

Multimodal user session dataset for RAG and search

Large Chinese dataset (1M to 10M) for search, recommendations, and RAG systems.

  • Question answering
  • Text classification
  • Sentence similarity
  • Chinese
5.3K 441M–10M rows
Datasetopendatalab/ChartVerse-SFT-1.8MTop pick

Chart and diagram reasoning SFT dataset

1.8M chart QA pairs with Chain-of-Thought annotations for VLM training.

  • Visual question answering
  • Image text → text
  • English
5.1K 1411M–10M rows
Datasetearakely-scale/CuaDriverBench-AgentEnvTop pick

AI agent benchmark for desktop multitasking with humans

Environment and execution logs for evaluating AI agents operating a computer alongside a human user.

  • Reinforcement learning
  • Image text → text
  • English
4.8K 2100K–1M rows
Datasetnvidia/describe-anything-datasetTop pick

Detailed photo and video object descriptions from NVIDIA

English multimodal dataset for training models to describe specific regions in images and videos in detail.

  • Image → text
  • Video text → text
  • English
4.5K 60100K–1M rows
DatasetTeklia/IAM-line

English handwritten text lines for OCR

A classical dataset of individual handwritten text line images in English for OCR training.

  • Image → text
  • English
4.2K 3710K–100K rows
DatasetOpenDataArena/MMFineReason-Full-2.3M-Qwen3-VL-235B-ThinkingTop pick

2.3m reasoning examples from Qwen3-VL-235B

Raw corpus of 2.3 million multimodal tasks and reasoning chains (8.8 billion tokens) before quality filtering.

  • Visual question answering
  • Question answering
  • Text generation
  • English
4K 651M–10M rows
Datasetnvidia/Llama-Nemotron-VLM-Dataset-v1Top pick

Llama-Nemotron VLM Multimodal Dataset

NVIDIA dataset for training and evaluating the Llama-Nemotron VLM multimodal model.

  • Visual question answering
  • Image text → text
  • Image → text
3.9K 1731M–10M rows
DatasetTIGER-Lab/VisualWebInstruct

Multimodal instructions for VLM reasoning

Approximately 900K question-answer pairs (40% image-based, 60% text-based) for boosting multimodal model reasoning logic.

  • Question answering
  • Visual question answering
  • Image text → text
  • English
3.9K 451M–10M rows
Datasetbiglam/british-library-book-images

One Million 16th-19th Century Book Illustrations

Over 1 million illustrations extracted from digitized historical books from the British Library (1510–1900).

  • Image classification
  • Image → text
  • Text → image
3.8K 651M–10M rows
DatasetScaleAI/HSSTop pick

Intuitive Visual Reasoning Benchmark

A collection of images and short videos with questions from Scale AI to evaluate models' ability to understand hidden context.

  • Visual question answering
  • Video text → text
  • English
3.8K 19up to 1K rows
Datasetbevaya/ScreenSpot

Text-based UI element grounding benchmark

Over 1,200 instructions and screenshots from iOS, Android, macOS, Windows, and the web for evaluating multimodal models.

  • Text generation
  • Image → text
  • English
3.7K 531K–10K rows
Datasetmaritaca-ai/enem

Brazilian ENEM 2022-2024 exam questions

Multiple-choice tasks from the Brazilian university entrance exam for evaluating LLMs and VLMs.

  • Visual question answering
  • Multiple choice
  • Portuguese
3.5K 18up to 1K rows
DatasetXkev/LLaVA-CoT-100k

100k VQA examples with step-by-step reasoning

VQA data with structured reasoning annotations for training Vision-Language Models.

  • Visual question answering
  • Image text → text
  • English
3.5K 10910K–100K rows
DatasetProGamerGov/synthetic-dataset-1m-dalle3-high-quality-captions

1M AI Images with High-Quality Captions

Over a million AI-generated images with detailed English captions for text-to-image and image-to-text tasks.

  • Text → image
  • Image classification
  • Image → text
  • English
3.4K 1621M–10M rows
DatasetUniParser/OmniScienceTop pick

Scientific images with text descriptions

A large multimodal dataset for understanding scientific images: images and text for image-to-text tasks.

  • Image → text
3.3K 1301M–10M rows🔒 on request
DatasetFBK-MT/MCIFTop pick

Multilingual multimodal benchmark on scientific reports

Human-annotated instruction dataset based on scientific reports: speech, video, and text across four languages to evaluate MLLMs.

  • Automatic speech recognition
  • Question answering
  • Summarization
  • English
  • German
  • Italian
  • +1
3.2K 711K–10K rows
Datasetvisual-layer/imagenet-1k-vl-enrichedTop pick

Enriched ImageNet-1K with descriptions and object boxes

An augmented version of ImageNet-1K with over a million records containing text descriptions, bounding boxes, and label error notes.

  • Object detection
  • Image classification
  • Text → image
  • English
3.1K 411M–10M rows
DatasetTencentARC/Plot2Code

Chart-to-code generation benchmark for MLLMs

Chart images and English texts to evaluate how multimodal models write code.

  • Text generation
  • Text → image
  • Image → text
  • English
3.1K 36up to 1K rows