800 kg air handler move proved by trial in a 3D physics model
ID workaround/task03 · revision 1 · results as of 17 November 2026
A crew of two must move a new 800 kg air handling unit from a hospital store room up to its plinth in the plant room, unmarked. The agent, as the rigging engineer, must prove the move in a 3D model of the site and write the method statement.
- Checks
- 18, 1 point each; the best trial counts
- Tested
- 7 models, one attempt each
- Best
- 44% (GPT‑6 Astra and Opus 5.5)
- Median
- 33%
- Full solves
- 0 of 7
- Median attempt
- 29 min, 165 tool calls
An 800 kg unit up a ramp to its plinth, unmarked, proved in a 3D model
Engineering design
By trial in a physics model, the agent must prove how two workers can get an 800 kg unit up a ramp unharmed.
Stateful tool use
The unit's supports, loads and tilt carry from move to move, so the steps must come in an order that does no harm.
Unstated requirements
The brief gives the goal and three conditions but no method; the workers' safe pushing and lifting limits have to be found.
A 3D model of the site, a kit store and two workers
The agent's world
A shell in a container with one command to the 3D world: walk, look, measure, place kit, direct the workers and reset.
The workers
Two workers played by code carry out commands within their cards' safe limits; the same command in the same state gets the same reply.
Limits
One session of up to 120 minutes and $20 at list prices; a reset starts a fresh trial, and the best trial counts.
The reward
18 checks from code: seven stages reached, and reached unharmed, plus facing, walkway and statement; claiming it seated when it isn't scores 0.
No attempt got the unit up the ramp; the best two reached 44%
Each dot is one attempt, and the line marks the median.
| Model | Reward |
|---|---|
| GPT‑6 AstraOpenAI | 44% |
| Opus 5.5Anthropic | 44% |
| Fable 5.2Anthropic | 33% |
| GPT‑6.1 SolOpenAI | 33% |
| Sonnet 5.5Anthropic | 33% |
| Sonnet 5Anthropic | 11% |
| GPT‑6 LunaOpenAI | 0% |
Given a step‑by‑step method, four of the seven met all 18 checks.
They never get up the ramp, stop with time left, and never turn the unit
Engineering design
No attempt brought the unit up to the plant room floor, so each lost at least 10 of the 18 checks.
86% handed in before an hour of their two had passed, though a reset makes a failed trial free.
Three called the packed runway over the ramp unsafe, too uncertain or unproved, and wrote a hold point into their statements.
71% never turned the unit end for end, so it couldn't have faced the right way on the plinth.
What an attempt costs, and what the environment includes
Cost
Attempts cost $0.24 to $18.67 at list prices for the agent model; the world and grader make no model calls.
What's in it
The 3D world and its client, the read‑only site code, all three versions and four variants, the grader, and a scripted reference walk.
Format
The agent runs in Docker and the 3D world in a headless browser beside it; packaged as a Harbor task on request.
Fresh variants
Other files, people and facts in the same design, made on request and never published.
The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.
Try one of our three sample environments
Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.
AsapiThe AI Agent Trials