AsapiRobot control
In these environments the agent is the controller of a simulated two‑handed robot. It moves the robot's joints with commands, and a physics engine decides what happens: a grip can slip, a screw can drop, a swing can miss.
A robot in a physics engine, one command at a time
- The robot
- Two robot arms. In the builds, each has seven joints and a five‑fingered hand that grips by friction alone, so a board stays held only while the fingers press on it. In batting, both hands hold the bat.
- Ten builds
- From carrying one beam into its outline to a dog‑house frame and roof from 19 boards and 90 screws, each driven with a hand‑held drill. The agent reads the scene's exact state and has 20 minutes.
- Batting
- The agent sees only a batter's‑eye camera image and moves the shoulders and elbows as ten pitches come in. The world keeps moving while the model thinks, at a fiftieth of real speed.
- No shortcuts
- The agent may not reset the scene, teleport a board or set its position directly. Every board has to be moved by the robot's hands.
- The score
- A build scores 100% when every board sits inside its outline, every joint has its three screws and everything is released and stable; otherwise it scores 0. Batting scores a point for each contact and one for each metre a fair hit carries.
- Along the way
- In the builds, the engine also returns a shaped reward after every action, based on how close the boards are to their targets. It doesn't count toward the score.
Near misses on placement, and screws that slip
Each attempt was given only the request.
Two boards placed together: GPT‑6.1 Sol carried both boards to their outlines, but they never sat fully inside, and its corrections kept disturbing them. They ended 4 and 9 mm out.
Screwed three‑board bridge: GPT‑6.1 Sol picked up the drill, but its fingers never kept hold of a screw, so neither joint was fixed. In the seven builds that need screws, none went in.
Screwed nine‑board grid with long rails: both long rails slipped from GPT‑6.1 Sol's grip while being carried, and time ran out before any board was in place.
Baseball batting from batter‑eye images: in the four models' attempts, the bat touched 10% of the pitches, every touch a foul, and never a curveball or a changeup.
GPT‑6.1 Sol completed none of the ten builds
Each dot is one attempt, given only the request; a build scores all or nothing. So far GPT‑6.1 Sol is the only model we've tested on the builds, once each, and the batting results cover four models.
Hover a dot for its model and reward. Batting has no top score; we show 1,000 points as 100%.
What each one asks the robot to do
| Environment | How it tests this | Checks | Models | Best |
|---|---|---|---|---|
| Flat‑beam transfer by a bimanual robot with friction‑only gripsRobot manipulation | Lift one loose beam, carry it about 15 cm and set it down with every corner inside its outline. | 1 | 1 | 0% |
| Three angled boards turned to flat outlines by a bimanual robotRobot manipulation | Pull three boards out of angled cradles, turn each flat and set it inside its outline, all three stable together. | 1 | 1 | 0% |
| Two boards placed together by a bimanual robot, one per handRobot manipulation | One board in each hand, both carried to their outlines and released, within a fixed budget of simulated time. | 1 | 1 | 0% |
| Screwed two‑board lap joint by a bimanual robot with a hand‑held drillRobot manipulation | Place two boards across each other, then fix them with three screws, each pinched, lined up with the drill and driven in. | 1 | 1 | 0% |
| Screwed three‑board corner frame by a bimanual robotRobot manipulation | Lay three boards flat as a U‑shaped corner and fix both overlaps with three screws each. | 1 | 1 | 0% |
| Screwed three‑board bridge by a bimanual robotRobot manipulation | Turn and place two supports and a span as a small bridge, then fix both overlaps with three screws each. | 1 | 1 | 0% |
| Screwed five‑board grid from angled stock by a bimanual robotRobot manipulation | Take five boards out of angled stock into a flat grid and fix all six overlaps, 18 screws in all. | 1 | 1 | 0% |
| Screwed nine‑board grid with long rails by a bimanual robotRobot manipulation | Two long rails and seven crosspieces, with larger robot arms on movable bases: 14 joints and 42 screws. | 1 | 1 | 0% |
| Upright slatted bench from 11 boards by a bimanual robotRobot manipulation | An upright slatted bench from 11 boards, bracing loose pieces with one hand while the other drives 42 screws. | 1 | 1 | 0% |
| Dog‑house frame and roof from 19 boards by a bimanual robotRobot manipulation | A dog‑house frame and flat roof from 19 boards, built in subframes, with 30 joints and 90 screws. | 1 | 1 | 0% |
| Baseball batting by a bimanual robot from batter‑eye imagesVisual robot manipulation | Bat against ten pitches of four kinds, moving only the shoulders and elbows, while the world keeps moving during each decision. | Points | 4 | 2 points |
Checks: each build has one, scored all or nothing; batting is scored in points.
Try a sample environment
Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.