AsapiThe AI Agent Trials
Asapi / Catalogue / Flat‑pack assembly manuals with 3D views from unlabelled photos
Environment datasheet · Household

Flat‑pack assembly manuals with 3D views from unlabelled photos

ID laycode/task12 · revision 1 · results as of 3 November 2026

Someone who has just moved has ten pieces of flat‑pack furniture, no instructions and a seller who doesn't answer. From photos alone, the agent builds one offline app: parts and hardware lists, small numbered steps, and at each step a 3D model of the assembly so far.

Checks
173, 60 manual points and 40 app points
Tested
7 models, one attempt each
Best
63% (Opus 5.5)
Median
53%
Full solves
0 of 7
Median attempt
33 min, 101 tool calls
What it tests

Ten assembly manuals with 3D views, from photos with no labels

Visual spatial understanding

From unlabelled photos, the agent counts each piece's panels and fittings, works out how they join, and shows the assembly in 3D.

Unstated requirements

Over half the requirements aren't in the request, such as distinct part names, placing hidden parts and a wall‑fixing step.

Agentic coding

Opened from a new folder, the app must load promptly using only its own or public files, and show a 3D view.

How it works

177 photos, none labelled, and an operator who walks every step

The agent's world

A fresh container with 177 photos, 14 to 25 per piece and none labelled, worked through the model's usual coding agent.

The photos

Rendered from 3D models of the makers' designs to look like phone photos: laid‑out panels, their reverse faces, hardware and each built piece.

Grading

An operator model walks every step of each manual in a fresh session, given the finished piece's photo and the answer key's parts.

The reward

173 checks: the manuals count 60 points and the app 40, with partial credit for the share of parts, hardware and joins found.

Results

Five of the seven scored 45–57%; the best reached 63% and the lowest 21%

Each dot is one attempt, and the line marks the median.

MedianFully solved
7 modelsOne attempt each53%0 of 7
ModelReward
Opus 5.5Anthropic63%
Fable 5.1Anthropic57%
Sonnet 5.5Anthropic54%
GPT‑6 AstraOpenAI53%
GPT‑6.1 SolOpenAI53%
Sonnet 5Anthropic45%
GPT‑6 LunaOpenAI21%

With an IT specialist's 22 requirements added to the request, the best scored 64%, against 63% with the request alone.

What models get wrong

They miscount hardware and panels, miss most joins, and reuse one cabinet

Visual spatial understanding

  1. All 70 manuals had wrong hardware counts, taken from the agent's own steps or a template, estimated, or left blank.

  2. 65 of the 70 manuals had wrong panel counts; at best, an attempt got 2 of its 10 pieces right.

  3. Every scored piece fell short of the joins pass line, and the strongest attempt found under a third of the real joins.

  4. With so few joins found, 65 of the 70 manuals left hidden parts unplaced, and 63 failed the build‑order check.

  5. Four of the seven attempts built most pieces from one shared cabinet template.

Running it

What an attempt costs, and what the environment includes

Cost

The median attempt cost about $7.75 at list prices for the agent model; the operator's and judges' calls are extra.

What's in it

The 177 photos, the answer key's parts, hardware and joins, the task in three versions, the full grader, and per‑check results.

Format

Packaged as a Harbor task on request; it builds in Docker, and grading opens the app in Chromium.

Fresh variants

Other files, people and facts in the same design, made on request and never published.

The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.