AsapiThe AI Agent Trials
Asapi / Capabilities / Visual spatial understanding
Capability · 8 environments

Visual spatial understanding

In these jobs, the sizes, positions and shapes that decide the result are only in pictures. Doing them well means measuring from what a photo shows, reading shapes exactly, and getting positions right.

What it takes

Three things an agent has to get right

Measure from the picture

Sizes and distances have to be read from what a picture shows, such as a floor tile, a door or a shadow.

Read the details exactly

Look‑alikes differ by colour, size or legs, and flat‑pack panels and fittings have to be counted, so small details decide it.

Get positions right

Pieces have to stand where the photos show, face the right way, and keep clear of door swings and heaters.

What models get wrong

They mismeasure, misread details, or put things in the wrong place

Measured wrong

  1. Room furnishing from photos: only 17% of attempts got the room's width right; most rounded the room to whole or half floor tiles, too narrow, with the ceiling too low.

  2. Flood‑claim replacement from dated photos: 63% of attempts bought the loveseat a size smaller than the one that stood in the room before the flood.

Details misread

  1. Flat‑pack assembly manuals: every attempt miscounted the screws and fittings on every piece, and the best found under a third of the real joins.

  2. Flood‑claim replacement from dated photos: 96% of attempts bought the wrong bench, most often in a lighter colour than in the photos.

Misplaced

  1. Room furnishing from photos: no attempt placed every piece within 30 cm of where it stands and facing the right way; attempts that finished got between 1 and 16 of the 17 pieces in place.

  2. Family move‑in from listing photos: in 92% of attempts, the layout stood a piece on a heating register or the heater, which show only in the listing photos.

Results

Partial credit for most attempts; the only full solves are finished race laps

Each dot is one attempt at the whole job, and the line marks the median. Larger dots are full solves, scoring 100 out of 100.

Hover a dot for its model and reward.

The 8 environments

How each environment tests it

EnvironmentHow it tests thisChecksModelsBest
Flat‑pack assembly manuals with 3D views from unlabelled photosHouseholdDetails. From unlabelled photos, the agent counts each piece's panels and fittings, works out how they join, and shows the assembly in 3D.173763%
Room furnishing from photos: measure, match look‑alikes, order and planFurnishing serviceMeasuring. The agent measures the room from six photos, with floor tiles and a door as the only scale, and places 17 pieces in it.28880%
Flood‑claim replacement from dated photos: what stood, what stays, look‑alikes and set‑upInsurance replacementDetails. Photos from three dates show the rooms changing before the flood, and each like‑for‑like replacement has to match its colour, size and base.25885%
Family move‑in from listing photos and papers: rooms, look‑alike purchases and full layoutsMove managementPositions. The listing photos show the steps and heaters, the plans the doors and their swings, and every room has to fit around them.45892%
Row‑house rooftop solar designed and quoted from plans, imagery and photosHome solarMeasuring. The roof and the neighbours that shade it are modelled from overhead images, phone photos and plans, with heights read from shadows.15699%
Baseball batting by a bimanual robot from batter‑eye imagesVisual robot manipulationMeasuring. Each pitch's position and speed have to be judged from a batter's‑eye image alone, with no numbers given.Points42 points
Four‑car race driven from cockpit images on a live physics clockMotor racingPositions. Each driver is given only its own cockpit image to judge its place on the road, its heading and the next bend.14100%
Four‑car race on a live physics clock with lasting crash damageMotor racingPositions. The same cockpit view as the first race, where each misjudged bend can now damage the car for good.14100%

Batting is scored in points, and a race all or nothing. Each environment's datasheet shows how its checks become the reward.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.