AsapiThe AI Agent Trials
Asapi / Capabilities / Robot control
Capability · 13 environments

Robot control

Here the agent controls a machine by command while a physics engine decides what happens: a two‑handed robot that builds and bats, or a race car driven from its cockpit view. Doing that well means moving precisely, handling objects through contact, and getting the order and timing right.

What it takes

Three things an agent has to get right

Move precisely

Goals must become movements that land where they should, a board within millimetres or a car on the road; small errors add up.

Handle objects through contact

Robots move only their own hands; objects move by contact, held by friction, and slip or shift when force or support is off.

Get the order and timing right

Steps must come in a workable order and at the right moment, from bracing a piece before fastening it to swinging or steering in time.

What models get wrong

They fall millimetres short, lose their grip, or mistime their moves

So far, one model has been tested on the builds, once each, four have a complete batting attempt, and four drove each race.

Falling short

  1. Two boards placed together: in the one attempt so far, both boards reached their outlines but never sat fully inside; corrections kept disturbing them, and they ended 4 and 9 mm out.

  2. Flat‑beam transfer: in the one attempt so far, the beam ended 16 mm outside its outline at its worst corner.

  3. Four‑car race on a live physics clock with lasting crash damage: GPT‑6 Luna crashed 21 times, until its engine failed before the end of the lap; the other three finished.

Losing grip

  1. Screwed three‑board bridge: in the one attempt so far, the fingers never kept hold of a screw, so neither joint was fixed; across the seven builds that need screws, not one screw went in.

  2. Screwed nine‑board grid with long rails: in the one attempt so far, both long rails slipped from the robot's grip while being carried, and time ran out before any board was placed.

Mistimed

  1. Baseball batting from batter‑eye images: across the four models' attempts, the bat touched 10% of the pitches, every touch a foul, and none on a curveball or a changeup.

Results

Zero on the builds, near zero at batting, while most race attempts finished their lap

Each dot is one attempt at the whole job; a build or a race scores all or nothing.

Hover a dot for its model and reward. Batting has no top score; we show 1,000 points as 100%.

The 13 environments

How each environment tests it

EnvironmentHow it tests thisChecksModelsBest
Flat‑beam transfer by a bimanual robot with friction‑only gripsRobot manipulationPrecision. The robot lifts one loose beam, carries it about 15 cm and sets it down with every corner inside its outline.110%
Three angled boards turned to flat outlines by a bimanual robotRobot manipulationPrecision. Three boards come out of angled cradles, each turned flat and set inside its outline, all three stable together.110%
Two boards placed together by a bimanual robot, one per handRobot manipulationPrecision. Each hand carries one board to its outline, and both must be released in place within a set amount of simulated time.110%
Screwed two‑board lap joint by a bimanual robot with a hand‑held drillRobot manipulationHandling. Two boards are laid across each other and fixed with three screws, each pinched, lined up with the drill and driven in.110%
Screwed three‑board corner frame by a bimanual robotRobot manipulationHandling. Three boards are laid flat as a U‑shaped corner, and both overlaps are fixed with three screws each.110%
Screwed three‑board bridge by a bimanual robotRobot manipulationHandling. Two supports and a span are set as a small bridge, the span on top, and both overlaps fixed with three screws each.110%
Screwed five‑board grid from angled stock by a bimanual robotRobot manipulationHandling. Five boards come out of angled stock into a flat grid, and all six overlaps need three screws each.110%
Screwed nine‑board grid with long rails by a bimanual robotRobot manipulationHandling. The robot carries two long rails and seven crosspieces from a stock fixture into a grid of 14 joints and 42 screws.110%
Upright slatted bench from 11 boards by a bimanual robotRobot manipulationHandling. The robot builds an upright slatted bench from 11 boards, one hand bracing loose pieces while the other drives 42 screws.110%
Dog‑house frame and roof from 19 boards by a bimanual robotRobot manipulationHandling. The robot builds a dog‑house frame and flat roof from 19 boards, driving 90 screws across 30 joints.110%
Baseball batting by a bimanual robot from batter‑eye imagesVisual robot manipulationTiming. Against ten pitches of four kinds, the agent swings using only the shoulders and elbows, while the world keeps moving.Points42 points
Four‑car race driven from cockpit images on a live physics clockMotor racingTiming. Each input holds for up to five seconds while the car moves on, so steering has to allow for a picture already seconds old.14100%
Four‑car race on a live physics clock with lasting crash damageMotor racingPrecision. Crashes permanently weaken the car and enough of them retire it, so the inputs have to keep it off the walls.14100%

Checks: each build and each race has one, scored all or nothing; batting is scored in points.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.