Dataset · jat-project
Multimodal data for a general agent
jat-project/jat-dataset
A mix of expert RL demonstrations, image-caption pairs, and text for training a multimodal agent.
What's inside
The dataset combines multiple separate sets: expert RL agent demonstrations, image-caption pairs, text data, and more. This is part of the JAT project to create a general-purpose multimodal agent. Time series are also specified in the metadata. The total size ranges from 100 million to 1 billion examples.
What you can do with it
- Train a multimodal agent with reinforcement learning
- Fine-tune VLMs on image-text pairs
- Evaluate QA and text generation
- Tasks
- Reinforcement learning, Text generation, Question answering
- Size
- 100M–1B rows
- Downloads this month
- 128.3K
- Likes
- 89
How to load it
from datasets import load_dataset
ds = load_dataset("jat-project/jat-dataset")