Skip to content

Dataset · ibm-research

IBM VAKRA benchmark for evaluating AI agents with API calls

ibm-research/VAKRA

Executable benchmark for testing complex multi-step agent reasoning when working with APIs and documents.

What's inside

A text dataset of 1,000 to 10,000 English examples from IBM Research. It represents multi-step dialogues and complete execution traces simulating a corporate environment. Unlike simple tests, VAKRA checks the ability of AI agents to combine tool-calling and document search for end-to-end problem solving.

What you can do with it

  • Evaluation and comparison of agent systems in multi-step API calling and knowledge base search (RAG)
  • Analysis of reasoning trajectories and debugging of end-to-end corporate AI assistants
Tasks
Question answering, Text retrieval, Text generation
Languages
English
Size
1K–10K rows
Downloads this month
6.7K
Likes
51

Models trained on it

How to load it

from datasets import load_dataset

ds = load_dataset("ibm-research/VAKRA")