Skip to content

Dataset · LLaVA-VL

Multimodal Instructions for Tool-Using AI Agents

LLaVA-VL/llava-plus-data

117K synthetic multimodal dialogues and instructions for training visual agents in tool use.

What's inside

The LLaVA-Plus-v1-117K dataset includes between 100,000 and 1 million examples (around 117,000) for visual and textual question-answering in English. Data was generated in September 2023 using the ChatGPT and GPT-4-0314 APIs for the LLaVA-Plus project.

What you can do with it

  • Training multimodal agents capable of invoking external tools
  • SFT fine-tuning of vision-language models (VLM) for instruction following
  • Testing models in visual question-answering tasks using tools
Tasks
Visual question answering, Question answering
Languages
English
Size
100K–1M rows
Downloads this month
86
Likes
36

How to load it

from datasets import load_dataset

ds = load_dataset("LLaVA-VL/llava-plus-data")