AsapiVisual spatial understanding
The facts that decide these jobs are in pictures: a room's size in a few photos, the parts of flat‑pack furniture laid out on a floor, a neighbour's roof height in a shadow seen from above. The agent has to measure, match and place things from what it sees.
The answer is in the pictures
- Rooms from photos
- In three furnishing jobs, the rooms and their furniture are seen in photos rendered from 3D scenes, with a sketch or plans in two of them. The agent works out what stands where, finds each piece among look‑alikes in web shops, orders it and says where everything goes.
- Measured by eye
- In the room‑furnishing job, the room's size and its doors and windows must be measured from the photos, with the floor tiles and the front door as the only scale.
- Placement
- In the room‑furnishing and flood‑claim jobs, each piece placed within 30 cm of where it really stands earns credit, and a separate share goes to pieces facing the right way.
- Flat‑pack from photos
- From 177 unlabelled photos of ten flat‑pack pieces, the agent builds a manual for each, with a 3D view at every step. The parts, fittings and joins have to be worked out from the photos.
- Roofs from above
- For a solar quote, the family's roof and the neighbours that shade it exist only in overhead images, phone photos and plans. Heights come from shadows.
- A batter's view
- A robot batter sees each pitch only as a camera image, with no numbers for the ball's position or speed.
- Who checks
- Code checks the furnishing jobs, the solar design and the batting. In the flat‑pack manuals, a model works through each manual in a browser, and scripts decide most checks from what it records and from the saved 3D scenes.
Measurements rounded, and look‑alikes bought a step off
Each attempt was given only the request.
Room furnishing from photos: more than half of the finished plans drew the room in whole or half floor tiles, and most made it too narrow, with the ceiling too low. One attempt in six measured the width closely enough for full credit.
Flood‑claim replacement from dated photos: most attempts that ordered found the right pieces but bought them a step off: the bench in a lighter colour, the loveseat a size smaller.
Family move‑in from listing photos: of what was bought for three pieces the family wanted in the same size as before, half was the wrong size, nearly all of it smaller than in their photos.
Flat‑pack assembly manuals: the best attempt found under a third of the real joins, and every attempt got the hardware counts wrong on every piece. Four of the seven built most pieces from one shared cabinet template.
Row‑house rooftop solar: two of the six attempts measured a neighbour's added floor from its shadow in the overhead images, then chose their design with the older plan's height.
Highest scores on rooms and roofs, lowest on flat‑pack manuals and batting
Each dot is one attempt, given only the request; the line marks the median, and larger dots are full solves.
Hover a dot for its model and reward. Batting has no top score; we show 1,000 points as 100%.
What each one asks the agent to work out from images
| Environment | How it tests this | Checks | Models | Best |
|---|---|---|---|---|
| Flat‑pack assembly manuals with 3D views from unlabelled photosHousehold | Count each piece's panels and fittings and work out how they join, all from photos, then show the assembly in 3D. | 173 | 7 | 63% |
| Room furnishing from photos: measure, match look‑alikes, order and planFurnishing service | Measure the room and its openings from six photos, buy the 17 pieces among look‑alikes, and plan where each stands. | 28 | 8 | 80% |
| Flood‑claim replacement from dated photos: what stood, what stays, look‑alikes and set‑upInsurance replacement | Work out from photos on three dates what stood before the flood, buy each piece's exact colour and size, and say where it goes. | 25 | 8 | 85% |
| Family move‑in from listing photos and papers: rooms, look‑alike purchases and full layoutsMove management | Read the house from listing photos and plans, match pieces to old‑home photos, and lay out every room around steps, heaters and doors. | 45 | 8 | 92% |
| Row‑house rooftop solar designed and quoted from plans, imagery and photosHome solar | Model the family's roof and the neighbours that shade it from overhead images, phone photos and plans, measuring heights from shadows. | 15 | 6 | 99% |
| Baseball batting by a bimanual robot from batter‑eye imagesVisual robot manipulation | Judge each pitch from a batter's‑eye image alone, with no numbers for the ball's position or speed. | Points | 4 | 2 points |
Checks: the number of requirements each grader scores; batting is scored in points.
Try a sample environment
Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.