AsapiThe AI Agent Trials
Asapi / Catalogue / Baseball batting by a bimanual robot from batter‑eye images
Environment datasheet · Visual robot manipulation

Baseball batting by a bimanual robot from batter‑eye images

ID llarms/task11 · revision 2 · results as of 10 November 2026

In a physics simulation running at a 50th of real speed, a two‑handed robot holds a bat. Seeing only the batter's camera view, the agent sets shoulder and elbow targets to hit ten mixed pitches, scoring for contact and for fair carry.

Checks
Points, 1 for contact plus 1 per metre of fair carry, over 10 pitches
Tested
4 models, one attempt each
Best
2 points (GPT‑6.1 Sol)
Median
1 point
What it tests

Ten pitches hit by a two‑armed robot, judged from the batter's eye

Visual spatial understanding

Each pitch's position and speed have to be judged from a batter's‑eye image alone, with no ball coordinates given.

Robot control

Against ten pitches of four kinds, the agent swings using only shoulder and elbow targets, while the world keeps moving.

How it works

A batter's‑eye camera, eight joint targets and a world that never pauses

The agent's world

Four tools, look, start, move and finish, through the model's usual coding agent, with its shell and web tools off.

What it sees

One 960 by 540 image from the batter's eyes per look or move, with the score and timing; no ball coordinates or joint readings.

Control

Eight joint targets, three at each shoulder and one at each elbow; the world runs on at a 50th of real speed, never pausing.

The reward

Over ten pitches, a contact earns a point, and a fair hit one more per metre of carry to its first bounce; there's no maximum.

Results

The four complete attempts made four contacts, every one foul

Each dot is one attempt, and the line marks the median.

MedianFully solved
4 modelsOne attempt each1 point–
ModelPoints
GPT‑6.1 SolOpenAI2 points
GPT‑6 AstraOpenAI1 point
GPT‑6 LunaOpenAI1 point
GPT‑6.1 TerraOpenAI0 points

Three more attempts stopped before the tenth pitch, Opus 5.5's and Fable 5.1's at an account limit and Sonnet 5.5's by its own choice; partial attempts get no score.

What models get wrong

They touch few pitches, all foul, and never a curveball or changeup

Robot control

  1. The four complete attempts touched 4 of their 40 pitches, all foul, so none earned carry; GPT‑6.1 Terra touched none of its ten.

  2. Of the 5 contacts across all 7 attempts, 4 came on sliders and 1 on a fastball; no curveball or changeup was touched.

  3. GPT‑6 Luna's one contact carried 79 m and GPT‑6 Astra's 30 m, but both went foul and earned a point each.

  4. Sonnet 5.5 ended its own attempt after two pitches, both missed, though its own output said the session was still live.

Running it

What an attempt takes, and what the environment includes

An attempt

The four complete attempts ran about 21 minutes each and sent 21 to 58 moves, typically 5 to 8 seconds after the image.

What's in it

The batting simulator and renderer with their tests, the tool server, the brief, the score checker, and all seven attempts' records, replays and videos.

Format

Runs on Node.js, with headless Chromium for the camera and Python for the tools; today on a Windows PC, with other packaging on request.

Fresh variants

Other pitch sequences in the same design, made on request and never published.

The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.