Robot control
Here the agent controls a machine by command while a physics engine decides what happens: a two‑handed robot that builds and bats, or a race car driven from its cockpit view. Doing that well means moving precisely, handling objects through contact, and getting the order and timing right.
Three things an agent has to get right
Move precisely
Goals must become movements that land where they should, a board within millimetres or a car on the road; small errors add up.
Handle objects through contact
Robots move only their own hands; objects move by contact, held by friction, and slip or shift when force or support is off.
Get the order and timing right
Steps must come in a workable order and at the right moment, from bracing a piece before fastening it to swinging or steering in time.
They fall millimetres short, lose their grip, or mistime their moves
So far, one model has been tested on the builds, once each, four have a complete batting attempt, and four drove each race.
Falling short
Two boards placed together: in the one attempt so far, both boards reached their outlines but never sat fully inside; corrections kept disturbing them, and they ended 4 and 9 mm out.
Flat‑beam transfer: in the one attempt so far, the beam ended 16 mm outside its outline at its worst corner.
Four‑car race on a live physics clock with lasting crash damage: GPT‑6 Luna crashed 21 times, until its engine failed before the end of the lap; the other three finished.
Losing grip
Screwed three‑board bridge: in the one attempt so far, the fingers never kept hold of a screw, so neither joint was fixed; across the seven builds that need screws, not one screw went in.
Screwed nine‑board grid with long rails: in the one attempt so far, both long rails slipped from the robot's grip while being carried, and time ran out before any board was placed.
Mistimed
Baseball batting from batter‑eye images: across the four models' attempts, the bat touched 10% of the pitches, every touch a foul, and none on a curveball or a changeup.
Zero on the builds, near zero at batting, while most race attempts finished their lap
Each dot is one attempt at the whole job; a build or a race scores all or nothing.
Hover a dot for its model and reward. Batting has no top score; we show 1,000 points as 100%.
How each environment tests it
| Environment | How it tests this | Checks | Models | Best |
|---|---|---|---|---|
| Flat‑beam transfer by a bimanual robot with friction‑only gripsRobot manipulation | Precision. The robot lifts one loose beam, carries it about 15 cm and sets it down with every corner inside its outline. | 1 | 1 | 0% |
| Three angled boards turned to flat outlines by a bimanual robotRobot manipulation | Precision. Three boards come out of angled cradles, each turned flat and set inside its outline, all three stable together. | 1 | 1 | 0% |
| Two boards placed together by a bimanual robot, one per handRobot manipulation | Precision. Each hand carries one board to its outline, and both must be released in place within a set amount of simulated time. | 1 | 1 | 0% |
| Screwed two‑board lap joint by a bimanual robot with a hand‑held drillRobot manipulation | Handling. Two boards are laid across each other and fixed with three screws, each pinched, lined up with the drill and driven in. | 1 | 1 | 0% |
| Screwed three‑board corner frame by a bimanual robotRobot manipulation | Handling. Three boards are laid flat as a U‑shaped corner, and both overlaps are fixed with three screws each. | 1 | 1 | 0% |
| Screwed three‑board bridge by a bimanual robotRobot manipulation | Handling. Two supports and a span are set as a small bridge, the span on top, and both overlaps fixed with three screws each. | 1 | 1 | 0% |
| Screwed five‑board grid from angled stock by a bimanual robotRobot manipulation | Handling. Five boards come out of angled stock into a flat grid, and all six overlaps need three screws each. | 1 | 1 | 0% |
| Screwed nine‑board grid with long rails by a bimanual robotRobot manipulation | Handling. The robot carries two long rails and seven crosspieces from a stock fixture into a grid of 14 joints and 42 screws. | 1 | 1 | 0% |
| Upright slatted bench from 11 boards by a bimanual robotRobot manipulation | Handling. The robot builds an upright slatted bench from 11 boards, one hand bracing loose pieces while the other drives 42 screws. | 1 | 1 | 0% |
| Dog‑house frame and roof from 19 boards by a bimanual robotRobot manipulation | Handling. The robot builds a dog‑house frame and flat roof from 19 boards, driving 90 screws across 30 joints. | 1 | 1 | 0% |
| Baseball batting by a bimanual robot from batter‑eye imagesVisual robot manipulation | Timing. Against ten pitches of four kinds, the agent swings using only the shoulders and elbows, while the world keeps moving. | Points | 4 | 2 points |
| Four‑car race driven from cockpit images on a live physics clockMotor racing | Timing. Each input holds for up to five seconds while the car moves on, so steering has to allow for a picture already seconds old. | 1 | 4 | 100% |
| Four‑car race on a live physics clock with lasting crash damageMotor racing | Precision. Crashes permanently weaken the car and enough of them retire it, so the inputs have to keep it off the walls. | 1 | 4 | 100% |
Checks: each build and each race has one, scored all or nothing; batting is scored in points.
Try one of our three sample environments
Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.
AsapiThe AI Agent Trials