AsapiEngineering design
In these environments the agent designs something physical that has to work: rooftop solar, a lighting plan, a way to move a heavy machine, or a floor repair it carries out in a simulation. Most of the reward is decided by testing it.
Simulation, physics or geometry decides most of the reward
- Two solar designs
- Rooftop solar for a tea factory and for a family's row house. The agent gets the simulator's formulas in a written engineering basis; the grader runs each design through it on weather years the agent never saw.
- Value against the best
- Half the reward is the design's ten‑year value as a share of the best design a search found. The other half is checks on the design and the figures the agent states for it.
- Hard limits
- Each solar design must keep to the job's rules, such as roof clearances, electrical limits and the client's budget. Breaking one forfeits the value half.
- A heavy machine
- In a 3D physics simulation of a building, the agent directs two workers to move an 800 kg air handler out of a store room, up a ramp and onto its plinth, turned to face the door, proving each step by doing it.
- Stage by stage
- Each of seven stages, from lifting the unit off the floor to seating it on its pins, counts once when reached and again when reached without harm, such as a scratch, a drop or an overloaded worker.
- A floor repair
- In a text‑based simulation of a condo, the agent finds a leak and rebuilds part of the floor with only what's in the unit, by 01:30. The simulation tracks each layer's height, so the grader can tell whether the floor stays flat and flush for six weeks.
- A lighting plan
- The agent writes a wiring guide for six smart ceiling lights a homeowner will fit himself. Two model judges read each light's position and type from the guide, and scripts check the geometry, such as whether a beam reaches the eyes of someone reading in bed.
Where the designs fall short
Each attempt was given only the request.
Tea factory rooftop solar: every attempt built more solar than pays, from an eighth more than the best design to nearly twice its size. Most of the extra power would be sold to the grid at a fifth of the price the factory pays.
Row‑house rooftop solar: GPT‑6.1 Sol measured the stair head 15 cm out, from the plot line instead of inside the parapet, and set its panels at the clearance's edge. That broke the placement rule, so its design, worth 94% of the best, lost the value half.
Night repair of a leaking condo floor: in 90% of attempts, the new floor wasn't flat and flush with the old one six weeks on, usually because the deeper strip by the door was packed with card or left low.
800 kg air handler move: every attempt fell short of getting the unit up the ramp to the plant room floor. Most handed in with more than half their time left, and three called the way over the ramp unsafe or uncertain.
DIY smart‑lighting plan: in six of the eight attempts, a light's beam reached the eyes of someone reading in bed. In two attempts, a wide beam crossed the ceiling fan's blades: those attempts checked only each light's centre against the fan.
One full solve, on the floor repair
Each dot is one attempt, given only the request; the line marks the median, and larger dots are full solves.
Hover a dot for its model and reward.
What each one asks the agent to design
| Environment | How it tests this | Checks | Models | Best |
|---|---|---|---|---|
| Tea factory rooftop solar and battery design with its business caseEnergy engineering | Size solar and a battery to cut the factory's bills, yet always hold enough to finish drying the tea when the grid fails. | 18 | 6 | 93% |
| 800 kg air handler move proved by trial in a 3D physics modelFacilities | Find how two workers can move an 800 kg unit out of a store room, along a corridor and up a ramp, proving each step. | 18 | 7 | 44% |
| DIY smart‑lighting plan from notes, photos and one shop's catalogueHousehold | Place six lights and wire their controls to light the room as the homeowner wants, with one shop's parts that work with his wiring. | 26 | 8 | 96% |
| Night repair of a leaking condo floor on an in‑world clockHousehold | Stop a leak and rebuild a floor from scarce materials so it stays flat and flush for six weeks, all in one night. | 13 | 7 | 100% |
| Row‑house rooftop solar designed and quoted from plans, imagery and photosHome solar | Design panels for one house among near‑identical ones, within the family's budget and payback limit, and quote figures the design really earns. | 15 | 6 | 99% |
Checks: the number of requirements each grader scores.
Try a sample environment
Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.