Room furnishing from photos: measure, match look‑alikes, order and plan

The request asks for an exact copy; the room and every piece exist only as pixels

Role

The assistant at a furnishing concierge, with the firm's trade accounts at three furniture web shops.

Request

Copy the model apartment's living‑dining room for the client, with the same pieces, sizes, colours and places, delivered by her date and within her budget, delivery included; order on the firm's card without waiting, and draw the floor plan.

Deliverable

The orders in the shops, with every live line, its delivery date and the total charged, and a floor plan as JSON and SVG: the room, its three openings and every piece with its position and facing.

What the reward depends on

The exact variant and count of each of 12 kinds, delivered in time and nothing extra; the room's size and openings close to true; each piece placed and facing close to where it stands; the budget; well‑formed files.

Where the requirements are

In six room photos, with the floor tiles' and front door's sizes, given in the request, as the only scale, and in the shops' product pages and photos. Nobody can be asked.

Six photos, three web shops and 305 product photo sheets

Interface

19 tools, among them a text browser (8 tools), a file tool (5) that writes only the plan's files and notes and checks their format, and the agent's image viewer. No shell, web or sub‑agents. Claude Code or Codex CLI on a Mac, effort high.

Photos

Six renders of the model apartment's room, 1,568 by 882 pixels with metadata removed, and 305 product photo sheets saved when the agent opens them.

Shops

Three furniture shops on one local site, two brand stores and one selling some of their pieces under the same names: 88 products in 320 variants, search and filters, carts, checkout that charges the card at once with one delivery fee per order, and cancellations.

Clock

None: a fixed date, Monday 1 February 2027, and a fixed delivery date for each product at each store.

Limits

One session of up to 150 minutes and 1,000 turns, with no spend cap; no attempt reached a limit.

100 points by code: purchase, floor plan, placement, budget and files

Structure

28 checks, 22 yes/no and six on a scale, worth purchase 35, floor plan 35, placement 20, budget 5 and files 5.

Purchase

Each of 12 kinds earns 2.5 for exactly the right variants in the right count, delivered in time, with no other variant of it; 5 more scale with the exact kinds, and any line outside the 12 kinds loses them.

Floor plan

Width and depth earn 10 each and height 2, in steps by error; each opening earns points for its place on its true wall and, with the others, 4 for their widths.

Placement

16 points share out by the pieces placed close to where they stand and 3 by fronts facing right, 1 for the plan's frame; positions are read rescaled to the true room or as written, the better counting.

Who decides

Code only, from the shops' final state and the two files; no model judges. The plan is laid on the room in whichever of eight orientations fits best. An independent second reading reproduced every stored grade.

Formula

Reward = points / 100. Budget points count only once the order covers 10 kinds and 14 units, and file points once the plan lists 12 items; ordering nothing and writing no plan scores 0.

The orders were mostly right, the room's size and the placement weren't, and none fully solved it

With only the request, the median attempt scored 63% and the best 80%, and every one of the 22 plans put some pieces in the wrong place. Given the measured floor plan and the list of pieces, both models tested scored 95% or more.

Each dot is one attempt; the line marks the median.

With only the request, each model made three attempts, and the table shows their mean. Told the rules, two models made one attempt each.

Told the rules: the request's unmeasured brochure sketch and the stager's refusal to share suppliers are replaced by the developer's measured floor plan and the stager's list of every piece (brand, model, colour, size, count), with no product codes, shops or positions. Both models lost points only on placement.

With only the request, the spread between a model's three attempts ran from 5 points (Sonnet 5.5) to 32 (Opus 5).

What the models got wrong

Counted over the 24 attempts with only the request, 8 models × 3, unless a line says otherwise.

1. Pieces out of place. Every one of the 22 plans put some pieces in the wrong place or facing the wrong way; they placed 1 to 16 of the 17 close enough to count.

2. The room by tile count. Full points on the width in 4 of 24 attempts, the depth in 9 and the ceiling height in 2. Of the 22 finished plans, 16 drew the width too short and 18 the ceiling too low.

3. Look‑alikes bought. The wrong variant or count of the coffee table in 14 of 24 attempts, the rug in 13 and the sideboard in 12. Nothing arrived late in any finished attempt.

4. Over budget. 8 of 24 attempts spent more than the budget, 6 of them buying more dining chairs than the photos show.

5. Ordering first. Every finished attempt ordered before drawing any plan, and 20 of 22 ordered USD 741 to 6,251 of pieces that would go back; reported, not scored.

6. Stopping at once. 2 of 24 attempts stopped within two minutes with nothing ordered and no plan.

What an attempt costs

Length

With only the request, the median attempt took 16 minutes and about 178 tool calls (21 to 365 across the 24).

Cost

Recorded for 12 of the 24 attempts with only the request, the four Anthropic models', at list prices by Claude Code's own count; the OpenAI models' agent logs tokens but not cost. With half unrecorded, we give no median or range.

What you license

With a licence

Revision 1 in both versions: the shops and their data, the room photos and product sheets, the grader, the answer key, a reference solution that scores 100, and the grader's report on each of the 26 attempts, as baseline results.

Format

Today it runs on a Mac in each agent's own sandbox; packaged as a Harbor task on request.

Fresh variants

Other data, people and facts in the same design, on request. They're never published.

Public or not

The AI Agent Trials, our public benchmark, sums up the job and shows each model's score. The files, grader and answer key aren't published.

What we haven't measured, and what's uneven

• Ordering nothing scores 0, and nothing with a generic plan 8%. A generic room with the first search hit for each kind scores 14%; given the true counts and the true room rounded to whole feet, a naive buyer scores 34%, and with a typical default room 17 to 22%.
• Delivery dates are fixed, so the job tests choosing shops and orders, not reacting to stock or delays.
• Pieces bought and then cancelled cost nothing in the reward; the pieces that would go back are reported, not scored.
• Told the rules was tested on two models, once each.
• The room was rendered in Blender from our own 3D scene, so its answers are exact, and its photos cleaner than a real apartment's.
• Two scoring choices came after an informal pilot: positions read both ways, and a second size set for the bookcases. Served photos are darker than the renders on four near‑colour pairs, and one look‑alike's photos contradict its page.