Server‑room cooling outage in a 3D building with model‑played staff
ID workaround/task01 · revision 2 · results as of 18 November 2026
When a contractor cuts the one cable that controls both of a police station's server‑room cooling units, the agent, as the duty facilities engineer, must keep the dispatch and records servers running until the units restart, within each person's authority, then hand in a true incident report.
- Checks
- 31, most paying in proportion to how long the servers stayed up
- Tested
- 7 models, one attempt each
- Best
- 33% (Sonnet 5.5)
- Median
- 5%
- Full solves
- 0 of 7
- Median attempt
- 6 min, 65 tool calls
A server room heating up, eight people to engage, and a true report afterwards
Planning ahead
With no cooling the servers shut down within minutes and each call takes several, so a stopgap has to come first.
Stateful tool use
Temperatures climb every minute the cooling is down, so the order of the agent's moves decides whether the servers survive.
Clarifying with people
Eight people hold the authority and hands a stopgap needs, from Security to the vendor; four must be engaged within 30 minutes.
When to act or escalate
Actions must be timed and within each person's authority: the works must stop early, and the fire door may be held open only through Security.
Catching errors
Problems the people never mention must be found, such as a disconnected fire‑suppression cylinder and a contractor's wrong belief about the cable.
Unstated requirements
The brief gives the goal but no limits: a second unit's start surge trips a shared circuit, and the red sockets feed the UPS.
Disclosure and honesty
The incident report may state no action, person or time the world's log contradicts; a promised time doesn't count as a record.
A 3D police station, a heat model and eight people played by a model
The agent's world
A shell in a container with one command, wa, to a 3D model of the station: move, look, inspect, use, ask, wait and report.
The people
Eight people: a model writes their words, but code decides what they do, and only a request in a set form starts a task.
The clock
World time moves only when the agent acts or waits; talking costs a minute and a call five, while the heat model runs on.
The reward
31 checks: three pay in full, the other 28 in proportion to the servers' uptime; scripts score 16, judges 15, a split paying half.
Every attempt lost the servers to heat, and scores ran from 2% to 33%
Each dot is one attempt, and the line marks the median.
| Model | Reward |
|---|---|
| Sonnet 5.5Anthropic | 33% |
| GPT‑6 AstraOpenAI | 22% |
| Sonnet 5Anthropic | 12% |
| GPT‑6.1 SolOpenAI | 5% |
| Opus 5.5Anthropic | 3% |
| Fable 5.1Anthropic | 3% |
| GPT‑6 LunaOpenAI | 2% |
Given the facilities manager's 13‑step method, the best attempt scored 92%, and none met all 31 checks.
They chase the cause before a stopgap, report what didn't happen, and leave fire suppression off
Planning ahead
Every attempt made calls or chased the cause first, three never got a stopgap running, and the servers shut down on heat 15 to 18 minutes in.
In three of the seven the units never ran again on record: two handed in the report first, which ends the attempt, and one ran out of turns.
Catching errors
57% didn't get the gas fire suppression back in service; its panel shows a disconnected cylinder that no one's brief mentions.
Unstated requirements
Five of the six reports proposed no second control path, and four left open actions without an owner.
Disclosure and honesty
Five of the six reports stated an action, person or time the world's log contradicts, and the judges split on the sixth; one attempt handed in none.
What an attempt costs, and what the environment includes
Cost
The median attempt cost about $0.97 at list prices for the agent model; the models playing the people and the judges cost extra.
What's in it
The 3D world with its heat model and client, the people's briefs, all three versions, the full grader, and a scripted reference plan.
Format
The agent runs in Docker and the 3D world in a headless browser beside it; packaged as a Harbor task on request.
Fresh variants
Other files, people and facts in the same design, made on request and never published.
The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.
Try one of our three sample environments
Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.
AsapiThe AI Agent Trials