Dataset · lmms-lab
M4 multi-image instruction sets
lmms-lab/M4-Instruct-Data
Multi-image instruction datasets for training multimodal models.
What's inside
M4-Instruct is a multi-image dataset collected from public datasets or generated via the GPT-4V API. It is designed to train LMMs to work with interleaved multiple images, such as LLaVA-NeXT-Interleave. Language is English, size is from 100k to 1M examples; collected in April 2024 and released in June 2024. Tasks are visual-question-answering and question-answering.
What you can do with it
- Train LMMs on multi-image instructions
- Fine-tune LLaVA-NeXT-Interleave-like models
- Verify answers across multiple images
- Tasks
- Visual question answering, Question answering
- Languages
- English
- Size
- 100K–1M rows
- Downloads this month
- 4.9K
- Likes
- 78
Models trained on it
- HuggingFaceTB/SmolVLM2-500M-Video-InstructImage text → text 1.7M
- HuggingFaceTB/SmolVLM2-2.2B-InstructImage text → text 194.6K
- HuggingFaceTB/SmolVLM2-256M-Video-InstructImage text → text 44.7K
- HuggingFaceTB/SmolVLM2-2.2B-BaseImage text → text 2.4K
- mradermacher/SmolVLM2-2.2B-Instruct-i1-GGUFVideo text → text 1.1K
- mradermacher/SmolVLM2-256M-Video-Instruct-i1-GGUF 824
- mradermacher/SmolVLM2-500M-Video-Instruct-i1-GGUF 737
- mradermacher/SmolVLM2-2.2B-Instruct-GGUFVideo text → text 368
How to load it
from datasets import load_dataset
ds = load_dataset("lmms-lab/M4-Instruct-Data")