Flat‑pack assembly manuals with 3D views from unlabelled photos
ID laycode/task12 · revision 1 · results as of 3 November 2026
Someone who has just moved has ten pieces of flat‑pack furniture, no instructions and a seller who doesn't answer. From photos alone, the agent builds one offline app: parts and hardware lists, small numbered steps, and at each step a 3D model of the assembly so far.
- Checks
- 173, 60 manual points and 40 app points
- Tested
- 7 models, one attempt each
- Best
- 63% (Opus 5.5)
- Median
- 53%
- Full solves
- 0 of 7
- Median attempt
- 33 min, 101 tool calls
Ten assembly manuals with 3D views, from photos with no labels
Visual spatial understanding
From unlabelled photos, the agent counts each piece's panels and fittings, works out how they join, and shows the assembly in 3D.
Unstated requirements
Over half the requirements aren't in the request, such as distinct part names, placing hidden parts and a wall‑fixing step.
Agentic coding
Opened from a new folder, the app must load promptly using only its own or public files, and show a 3D view.
177 photos, none labelled, and an operator who walks every step
The agent's world
A fresh container with 177 photos, 14 to 25 per piece and none labelled, worked through the model's usual coding agent.
The photos
Rendered from 3D models of the makers' designs to look like phone photos: laid‑out panels, their reverse faces, hardware and each built piece.
Grading
An operator model walks every step of each manual in a fresh session, given the finished piece's photo and the answer key's parts.
The reward
173 checks: the manuals count 60 points and the app 40, with partial credit for the share of parts, hardware and joins found.
Five of the seven scored 45–57%; the best reached 63% and the lowest 21%
Each dot is one attempt, and the line marks the median.
| Model | Reward |
|---|---|
| Opus 5.5Anthropic | 63% |
| Fable 5.1Anthropic | 57% |
| Sonnet 5.5Anthropic | 54% |
| GPT‑6 AstraOpenAI | 53% |
| GPT‑6.1 SolOpenAI | 53% |
| Sonnet 5Anthropic | 45% |
| GPT‑6 LunaOpenAI | 21% |
With an IT specialist's 22 requirements added to the request, the best scored 64%, against 63% with the request alone.
They miscount hardware and panels, miss most joins, and reuse one cabinet
Visual spatial understanding
All 70 manuals had wrong hardware counts, taken from the agent's own steps or a template, estimated, or left blank.
65 of the 70 manuals had wrong panel counts; at best, an attempt got 2 of its 10 pieces right.
Every scored piece fell short of the joins pass line, and the strongest attempt found under a third of the real joins.
With so few joins found, 65 of the 70 manuals left hidden parts unplaced, and 63 failed the build‑order check.
Four of the seven attempts built most pieces from one shared cabinet template.
What an attempt costs, and what the environment includes
Cost
The median attempt cost about $7.75 at list prices for the agent model; the operator's and judges' calls are extra.
What's in it
The 177 photos, the answer key's parts, hardware and joins, the task in three versions, the full grader, and per‑check results.
Format
Packaged as a Harbor task on request; it builds in Docker, and grading opens the app in Chromium.
Fresh variants
Other files, people and facts in the same design, made on request and never published.
The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.
Try one of our three sample environments
Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.
AsapiThe AI Agent Trials