Skip to content

Dataset · Alibaba-VELLDEPTH

Multimodal Agent Safety Benchmark

Alibaba-VELLDEPTH/MMA-SafetyBench

Dataset and evaluations for checking the safety of multimodal agents on web and mobile tasks.

What's inside

Contains between 1k and 10k items, including text, tabular, and visual data. It is a benchmark featuring 1,083 adversarial trajectories covering tasks for Web, desktop GUI, Mobile, Document, and Video.

What you can do with it

  • Evaluating the safety of multimodal agents and systems
  • Benchmarking the robustness of LLM models against adversarial attacks
Size
1K–10K rows
Downloads this month
54.4K
Likes
20

How to load it

from datasets import load_dataset

ds = load_dataset("Alibaba-VELLDEPTH/MMA-SafetyBench")