Dataset · ibm-research
IBM VAKRA benchmark for evaluating AI agents with API calls
ibm-research/VAKRA
Executable benchmark for testing complex multi-step agent reasoning when working with APIs and documents.
What's inside
A text dataset of 1,000 to 10,000 English examples from IBM Research. It represents multi-step dialogues and complete execution traces simulating a corporate environment. Unlike simple tests, VAKRA checks the ability of AI agents to combine tool-calling and document search for end-to-end problem solving.
What you can do with it
- Evaluation and comparison of agent systems in multi-step API calling and knowledge base search (RAG)
- Analysis of reasoning trajectories and debugging of end-to-end corporate AI assistants
- Tasks
- Question answering, Text retrieval, Text generation
- Languages
- English
- Size
- 1K–10K rows
- Downloads this month
- 6.7K
- Likes
- 51
Models trained on it
How to load it
from datasets import load_dataset
ds = load_dataset("ibm-research/VAKRA")