Room furnishing from photos: measure, match look‑alikes, order and plan
ID shop‑the‑room/task01 · revision 1 · results as of 3 November 2026
For a furnishing concierge, the agent copies a model apartment's room that the client knows only from six photos. It measures the room, finds every piece among look‑alikes in three web shops, orders within budget to arrive in time, and draws the delivery crew's floor plan.
- Checks
- 28, 100 points in five parts
- Tested
- 8 models, 3 attempts each
- Best attempt
- 80% (GPT‑6 Astra)
- Median
- 63%
- Full solves
- 0 of 24
- Median attempt
- 16 min
A room copied from six photos: measured, bought among look‑alikes and drawn
Visual spatial understanding
The agent measures the room from six photos, with floor tiles and a door as the only scale, and places 17 pieces in it.
Stateful tool use
Orders charge at once, and the delivery fee comes back only if a whole order is cancelled, so a hasty order costs money.
Planning ahead
Three pieces arrive in time from only one shop, and each order adds a delivery fee, so how the orders are split decides the cost.
Six photos, three web shops and a card that charges at once
The agent's world
A text browser into three local furniture shops, a file tool for the floor plan and the agent's image viewer; no shell or web.
The shops
88 products in 320 variants across two brand stores and one reseller, with search, carts, checkout and cancellations; each order pays one delivery fee.
Limits
One session of up to 150 minutes and 1,000 turns, with nobody to ask; no attempt reached a limit.
The reward
28 checks by code alone: the purchase and the floor plan 35 points each, placement 20, budget 5 and files 5.
Two‑thirds of attempts scored 50–80%; the rest scored 46% or less
Each dot is one attempt, and the line marks the median.
| Model | Mean of 3 attempts |
|---|---|
| GPT‑6 AstraOpenAI | 74% |
| Opus 5.5Anthropic | 73% |
| GPT‑6.1 SolOpenAI | 70% |
| Sonnet 5.5Anthropic | 62% |
| Fable 5.1Anthropic | 53% |
| Opus 5Anthropic | 48% |
| GPT‑6 SolOpenAI | 45% |
| GPT‑6 LunaOpenAI | 8% |
Given the measured floor plan and the list of pieces, both models tested scored 95% or more.
They misplace pieces, measure by tile count, and buy look‑alikes
Visual spatial understanding
Every one of the 22 finished plans put some pieces in the wrong place or facing the wrong way; they placed 1 to 16 of the 17 close enough to count.
Only 17% of attempts got full points for the room's width, 38% for its depth and 8% for its height; most plans drew it too narrow and too low.
58% bought the wrong variant or count of the coffee table, 54% of the rug and 50% of the sideboard.
Planning ahead
33% spent more than the budget, most of them by buying six dining chairs where the photos show four.
Stateful tool use
Every finished attempt ordered before drawing any plan, and 20 of the 22 bought $741 to $6,251 of pieces that would go back; this isn't scored.
What an attempt takes, and what the environment includes
An attempt
The median attempt used about 9 million tokens, cached input included.
What's in it
The shops and their data, the room photos and product sheets, both versions, the full grader, and a reference solution that scores 100.
Format
It runs today on a Mac in each agent's own sandbox, and is packaged as a Harbor task on request.
Fresh variants
Other files, people and facts in the same design, made on request and never published.
The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.
Try one of our three sample environments
Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.
AsapiThe AI Agent Trials