Upright slatted bench from 11 boards by a bimanual robot

An upright bench: 11 boards, 14 joints, 42 screws

Role

The controller of a simulated bimanual robot in a workshop of wooden boards, acting only through a local HTTP command service, with nobody giving movement advice.

Request

Pick up and place all 11 boards, turning them as needed, so every corner rests inside its outline, then fix all 14 joints with their 42 screws and back the drills out; check the measured result before stopping.

Deliverable

The final physical state, from the engine's last observation: 11 boards placed and released, 42 screws installed, all 14 joints rigid and the drills withdrawn. A short summary is saved too; its claims don't count.

What the reward depends on

Every board inside an outline of its own size once settled, after a real lift and release, all 14 joints rigid with three screws each, and the drills withdrawn. One point or none.

Where the requirements are

Stated in the request and one interface document, with the runtime's source open to read; each observation reports the targets and progress. Nothing is hidden: the difficulty is physical control.

A contact‑only physics workshop, read as exact state and driven by typed commands

Interface

A Codex chat subagent running GPT‑6.1 Sol at its default reasoning setting, sending JSON to a local HTTP control server through its shell tools: read the state, send a step or a sequence of velocity segments, and post short progress notes.

Physics

Rigid‑body physics (cannon‑es) in its own headless Edge process at 180 Hz, apart from any viewer. Simulated time advances only during commands and pauses while the agent plans.

The robot

Two seven‑joint manipulators on bases that slide across the floor, each with a palm and five fingers of three joints, with spread and thumb opposition. Motion is rate‑limited and stopped by collisions, so a target isn't always reached.

Grip

No attachment constraint: a hold is opposed, force‑bearing finger contact, so boards, drills and screws can slip, turn or drop while the hand stays closed, and over‑curling can push a board out.

Observations

Exact structured state, not images: board poses, sizes and velocities; drill and screw poses and states; finger angles, contact force and slip; collision and joint‑limit feedback; each target's pose and progress.

The build

On each side a foot runner, two upright legs and a seat support, then three seat slats across: 11 boards from angled stock, joined at 14 overlaps; unsupported pieces need bracing while fixed.

Drill and screws

Graspable drills with no magazine, and screws handled one at a time: pinch a screw, set it on the overlap, align the other hand's drill bit, engage, let go, feed along the axis, then withdraw before any sideways move.

Recording

Every command result and sampled pose is saved; four fixed cameras render review videos afterwards, at simulation speed. The saved observations, not the videos, are the record.

Network

Internet on, for the pinned physics libraries, and web tools available; nothing the job needs is online.

Limits

One session of 20 minutes of wall time, planning included, plus the scenario's own action and simulation‑time budgets. The request rules out resets, teleporting, direct state changes and earlier trajectories.

One all‑or‑nothing check on the final physical state

Structure

One check, worth one point: the whole build complete. Boards placed, screws installed and each board's worst corner protrusion are reported as diagnostics, never scored.

Placement

Each board must be lifted, released and left to settle with all eight corners inside a target outline of its own size, allowing only a numerical margin. Holding a board in place doesn't count.

Fastening

Every joint rigid, made by three non‑collinear screws driven through the top board into the one below, and every drill withdrawn. Matching the outlines alone doesn't pass.

Who decides

A script reads the final observation and transforms every board's eight corners into its target's frame; it also needs the engine's own success flag for lift, release, settling and any joints. No model judge, and the agent's claims don't count.

The rule in the request

The request stated that every corner must lie inside its outline, with the margin for rounding only, so this attempt was told the strict rule.

Engine reward

After each action the engine also returns a shaped reward from the boards' distances to their targets, with a bonus on success and a penalty for a lost board; the grader ignores it.

Formula

Reward = 1 if the build is complete, else 0. An untouched scene, a partial build or a missing final state scores 0.

Every screw it pinched slipped out on the next lift

GPT‑6.1 Sol, the only model tested on the builds, scored 0%. It moved slats, runners and legs, losing grips on the way, and held the drill from early on, but each of the four screws it pinched slipped on the next lift. No board ended inside its outline.

Each dot is one attempt; the line marks the median.

One attempt, by GPT‑6.1 Sol at seed 1; no other model has been tested on the ten builds.

Made on 2 November 2026 on its own control server and physics process, told the strict rule in the request. Its closing summary called the build incomplete, as the grader found.

What the model got wrong

One attempt, by GPT‑6.1 Sol, told the strict rule; command numbers are from its saved sequence.

1. Grips lost in transport. Commands 3 to 26 extracted slats, runners, legs and a seat support, with dropped grips and repeated collisions with stock.

2. Screws slipped on lifting. Holding the drill from command 28, it pinched a screw at commands 69, 109, 148 and 187; each grasp was gone after the next lift.

3. Half the commands on fine approaches. Commands 193 to 383 were short screw approaches and finger moves, with no further screw grasp.

4. Nothing in place. At the 20‑minute limit, 0 of 11 boards were inside their outlines, the closest 259 mm out, and 0 of 42 screws were in.

What an attempt costs

Length

The attempt used its full 20 minutes of wall time, with 383 motion commands, many of them short approaches, and 252 simulated seconds. Model calls, tool calls and tokens weren't recorded; a motion command isn't a model call.

Cost

Not recorded: the attempt ran on a subscription sign‑in, which logs no cost or tokens, so we give no figure.

What you license

With a licence

Revision 1: the frozen runtime with this scenario at seed 1, the request and interface document, the grader and its self‑test, and the attempt's commands, observations, pose trajectory and four review videos, as baseline results.

Format

Runs on Node.js, with the physics in headless Microsoft Edge driven by Playwright, and a Python grader; today on a Windows PC, outside a container. Other packaging on request.

Fresh variants

Other builds and layouts in the same design, on request. They're never published.

Public or not

The AI Agent Trials, our public benchmark, sums up the job and shows each model's score, so hold out fresh variants for evaluation. The files, grader and answer key aren't published.

What we haven't measured, and what's uneven

• One attempt, by one model, at one seed, so we haven't measured how much attempts vary or how other models do.
• No reference solution: no attempt, scripted or by an agent, has completed a build under the strict check, so it isn't shown that one can be; the self‑tests check only the scoring.
• Each outline is its board's own size, so a board must settle almost exactly at its target pose; whether the contact solver allows that hasn't been tested.
• The grader trusts the engine's own flag for lift, release, settling and joints and checks only the geometry; regraded copies with that flag forced true still failed.
• The agent reads exact state and may read the runtime's source, which reveals the thresholds: this tests control from state, not from vision.
• Simplified physics: rigid contacts, simplified fingers and idealised stock fixtures, with no thread cutting or motor torque, so results don't speak to real robots.
• Up to three attempts shared one PC at a time, so wall time may include contention.
• The engine's lift flag can register for a board grasped in raised stock without upward travel, so on its own it doesn't prove a lift.