Family move‑in from listing photos and papers: rooms, look‑alike purchases and full layouts
ID shop‑the‑room/task03 · revision 1 · results as of 12 November 2026
For a move manager, the agent gets a house ready for a family of six: who sleeps and works where, what to buy, some of it as before, which of their own pieces come along, and where every piece stands, within a budget and two delivery days.
- Checks
- 45, 100 points in seven groups
- Tested
- 8 models, 3 attempts each
- Best attempt
- 92% (GPT‑6 Astra)
- Median
- 83%
- Full solves
- 0 of 24
- Median attempt
- 19 min
Six rooms, a family of six, and the care the house calls for
Visual spatial understanding
Steps and heaters show only in 17 listing photos, and the plans draw doors and windows with room sizes but no other dimensions.
Unstated requirements
Three placement standards are written, and the brief says they aren't the whole job: pieces must also keep off heaters and clear of door swings.
Stateful tool use
Each order charges the card at once and must arrive on one of two days, within the family's budget.
17 listing photos, two plans, the family's papers and three shops
The agent's world
A text browser into three local furniture shops, a file tool for the plan and the agent's image viewer; no shell or web.
The papers
A family chat, discharge instructions, a night‑shift schedule, an allergy plan, vet records and a measured inventory of 27 pieces.
Limits
One session of up to 150 minutes and 1,000 turns, with nobody to ask; no attempt reached a limit.
The reward
100 points from code alone, across rooms, purchases, the family's own pieces, their needs, layout, money and the plan's form.
Most attempts scored 78–92%, and one stopped after 75 seconds
Each dot is one attempt, and the line marks the median.
| Model | Mean of 3 attempts |
|---|---|
| Opus 5.5Anthropic | 90% |
| GPT‑6 AstraOpenAI | 88% |
| GPT‑6.1 SolOpenAI | 86% |
| Opus 5Anthropic | 82% |
| Fable 5.1Anthropic | 82% |
| Sonnet 5.5Anthropic | 81% |
| GPT‑6 SolOpenAI | 70% |
| GPT‑6 LunaOpenAI | 37% |
Given the needs list, the site rules and a measured plan, both models tested scored 100%.
They block heaters and doors, buy sizes from the shop page, and buy beyond the list
Unstated requirements
92% of attempts put a piece on a heating register or the heater, and 71% put one in a door's swing.
Visual spatial understanding
Of 61 lines bought for the as‑before products whose size counts, half were the wrong size, most of those the size the product page says its photos show.
17% of attempts swapped two rooms' uses, putting someone in a room whose steps, shown only in the listing photos, rule it out.
Stateful tool use
92% of attempts bought units beyond the family's list, each a wrong size or a look‑alike of a piece wanted as before.
What an attempt takes, and what the environment includes
What's in it
The shops and their data, the house's photos and plans, both versions, the full grader, and a reference solution that scores 100.
Format
It runs today on a Mac in each agent's own sandbox, and is packaged as a Harbor task on request.
Fresh variants
Other files, people and facts in the same design, made on request and never published.
The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.
Try one of our three sample environments
Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.
AsapiThe AI Agent Trials