AsapiThe AI Agent Trials
Asapi / Catalogue / Night repair of a leaking condo floor on an in‑world clock
Environment datasheet · Household

Night repair of a leaking condo floor on an in‑world clock

ID workaround/task05 · revision 1 · results as of 17 November 2026

Arriving alone one evening to a soaked hall floor and a leak of unknown source, the agent, as the condo's owner, must stop the water, rebuild the floor to last some six weeks with only what is in the flat, tell the buyer and guest, and leave on time.

Checks
13, eight paid by the share of the wet strip opened, six only on a safe floor
Tested
7 models, 3 attempts each
Best attempt
100% (GPT‑6 Astra)
Median
41%
Full solves
1 of 21
Median attempt
2 min
What it tests

A leak to stop and a floor to rebuild, before the guest arrives

Stateful tool use

Each layer laid changes the floor's height, materials run out, and the clock runs to a fixed deadline.

Engineering design

The rebuilt floor must stay flat and flush for six weeks, with only what is in the flat, and the work done in time.

Catching errors

Two bathroom joints look unsealed but only one leaks, shown only by a coloured‑water test, and the board is wet past the stain.

Disclosure and honesty

The buyer must hear what was found and done, every disturbed room included, and no message may hide a known problem.

Unstated requirements

The brief asks for a safe repair lasting about six weeks; checks spell out what that takes, such as leaving no wet wood.

How it works

A text model of the flat, what's in the unit, and four and a half hours

The agent's world

A shell in a container with one command to the flat: look, feel, measure, test, dry, seal, lay floors, message and leave.

Hidden state

The leak's source, how far the wet goes and the hole's depths show only through the agent's own tests and measurements.

The clock

World time moves only when the agent acts or waits, by a published table of minutes; the night holds four and a half hours.

The reward

13 checks: scripts score ten on the final state, most paid by the share of the wet strip opened; judges read the messages.

Results

Scores ran from 0% to 100%, with a median of 41%, and one attempt met all 13 checks

Each dot is one attempt, and the line marks the median.

MedianFully solved
7 models3 attempts each41%1 of 21
ModelMean of 3 attempts
GPT‑6 AstraOpenAI87%
Fable 5.1Anthropic67%
GPT‑6.1 SolOpenAI62%
Fable 5.2Anthropic56%
Opus 5.5Anthropic44%
Sonnet 5.5Anthropic16%
GPT‑6 LunaOpenAI15%

Given every command of one passing method, all 20 scored attempts met all 13 checks.

What models get wrong

Their floors don't last, edges stay open, and time runs out

Engineering design

  1. 67% of attempts left a floor that isn't flat and flush after six weeks, and 48% one that isn't even flush on the night.

  2. 81% didn't seal the joint between the new floor and the old all the way along.

Stateful tool use

  1. 57% weren't finished and gone by the night's end, after minutes spent drying a base that won't dry or laying planks twice.

Disclosure and honesty

  1. 43% sent a message that was false or hid a known problem, such as an edge called sealed after the silicone ran out.

Running it

What an attempt costs, and what the environment includes

Cost

Attempts cost $0.06 to $1.45 at list prices for the agent model; the judges are extra, and the world makes no model calls.

What's in it

The world with its state machine and client, all three versions, the grader and the judges' fact‑sheet builder, and a scripted reference walk.

Format

The agent runs in Docker and the world in a second container; packaged as a Harbor task on request.

Fresh variants

Other files, people and facts in the same design, made on request and never published.

The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.