AsapiThe AI Agent Trials
Asapi / Catalogue / 800 kg air handler move proved by trial in a 3D physics model
Environment datasheet · Facilities

800 kg air handler move proved by trial in a 3D physics model

ID workaround/task03 · revision 1 · results as of 17 November 2026

A crew of two must move a new 800 kg air handling unit from a hospital store room up to its plinth in the plant room, unmarked. The agent, as the rigging engineer, must prove the move in a 3D model of the site and write the method statement.

Checks
18, 1 point each; the best trial counts
Tested
7 models, one attempt each
Best
44% (GPT‑6 Astra and Opus 5.5)
Median
33%
Full solves
0 of 7
Median attempt
29 min, 165 tool calls
What it tests

An 800 kg unit up a ramp to its plinth, unmarked, proved in a 3D model

Engineering design

By trial in a physics model, the agent must prove how two workers can get an 800 kg unit up a ramp unharmed.

Stateful tool use

The unit's supports, loads and tilt carry from move to move, so the steps must come in an order that does no harm.

Unstated requirements

The brief gives the goal and three conditions but no method; the workers' safe pushing and lifting limits have to be found.

How it works

A 3D model of the site, a kit store and two workers

The agent's world

A shell in a container with one command to the 3D world: walk, look, measure, place kit, direct the workers and reset.

The workers

Two workers played by code carry out commands within their cards' safe limits; the same command in the same state gets the same reply.

Limits

One session of up to 120 minutes and $20 at list prices; a reset starts a fresh trial, and the best trial counts.

The reward

18 checks from code: seven stages reached, and reached unharmed, plus facing, walkway and statement; claiming it seated when it isn't scores 0.

Results

No attempt got the unit up the ramp; the best two reached 44%

Each dot is one attempt, and the line marks the median.

MedianFully solved
7 modelsOne attempt each33%0 of 7
ModelReward
GPT‑6 AstraOpenAI44%
Opus 5.5Anthropic44%
Fable 5.2Anthropic33%
GPT‑6.1 SolOpenAI33%
Sonnet 5.5Anthropic33%
Sonnet 5Anthropic11%
GPT‑6 LunaOpenAI0%

Given a step‑by‑step method, four of the seven met all 18 checks.

What models get wrong

They never get up the ramp, stop with time left, and never turn the unit

Engineering design

  1. No attempt brought the unit up to the plant room floor, so each lost at least 10 of the 18 checks.

  2. 86% handed in before an hour of their two had passed, though a reset makes a failed trial free.

  3. Three called the packed runway over the ramp unsafe, too uncertain or unproved, and wrote a hold point into their statements.

  4. 71% never turned the unit end for end, so it couldn't have faced the right way on the plinth.

Running it

What an attempt costs, and what the environment includes

Cost

Attempts cost $0.24 to $18.67 at list prices for the agent model; the world and grader make no model calls.

What's in it

The 3D world and its client, the read‑only site code, all three versions and four variants, the grader, and a scripted reference walk.

Format

The agent runs in Docker and the 3D world in a headless browser beside it; packaged as a Harbor task on request.

Fresh variants

Other files, people and facts in the same design, made on request and never published.

The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.