Skip to content

Dataset · google

Google engineering task benchmark

google/physical-engineerings-last-exam

A set of complex multi-step tasks from licensed engineers for evaluating AI agents.

What's inside

It contains English engineering tasks across 4 disciplines created by licensed professionals. In each task, the agent must work in a workspace featuring blueprints, specifications, tables, and CAD/BIM tools to produce calculations, code, models, and reports. The repository features a dev set comprising fewer than 1,000 examples.

What you can do with it

  • Evaluating autonomous AI agents in complex engineering and CAD/BIM tasks
  • Testing model capabilities with blueprints, code, and calculations
  • Benchmarking multimodal LLMs for technical design
Tasks
Text generation, Question answering
Languages
English
Size
up to 1K rows
Downloads this month
378.3K
Likes
191
Access
on request on Hugging Face

How to load it

from datasets import load_dataset

ds = load_dataset("google/physical-engineerings-last-exam")