Planning ahead
Many choices only show their cost or value weeks or months later. Doing well means counting the cost over time, preparing ahead, and choosing by where an option leads, not how it looks today.
Three things an agent has to get right
Count costs and value over time
In several, a profit model prices each plan over months or years, so a choice that looks good now can lose money by the end.
Prepare ahead
Lead times, booking windows and seasonal peaks mean orders, shifts and tests have to be arranged before they're needed.
Choose by where it leads
The best option is often not the obvious one: a repair over a replacement, a smaller solar array, a course that opens up later terms.
They misjudge later value, start too late, or take the obvious option
Value misjudged
Tea factory rooftop solar: every attempt built a bigger solar array than pays off, up to nearly twice the size of the best design; most of the extra power sells to the grid at a fifth of the import price.
Household savings plan: 63% of attempts built the plan on uncertain weekend side income, with no version that works without it.
Not ready in time
Tea‑factory cash desk: in 40% of attempts, the agent asked the owner for money too late in one of the 16 scenarios, since money takes three weeks to arrive, and the business went under.
Coffee roaster materials planning: in 86% of attempts, the agent failed in at least one season to prepare for a large customer order that the commercial manager had warned it about.
The obvious choice
12‑month equipment plan: 69% of attempts bought a new cabinet when a cheaper repair would fix the old one.
Course registration optimised for earliest graduation: 93% of attempts kept the course already held in a contested slot and missed the one that would let the student graduate sooner.
Partial credit for most attempts, and full solves in six of the 21 environments
Each dot is one attempt at the whole job, and the line marks the median. Larger dots are full solves, scoring 100 out of 100.
Hover a dot for its model and reward.
How each environment tests it
| Environment | How it tests this | Checks | Models | Best |
|---|---|---|---|---|
| Cancelled‑flight family rebooking against a ticking clockTravel concierge | Ahead of time. The clock moves with every page, form and message, and alternative seats sell out at fixed times, so securing seats early pays. | 61–63 | 8 | 85% |
| Household solar purchase plan with simulated contactsHousehold | Cost over time. Paying routes, from cash to finance, card instalments or waiting, are scored over time against a rebate deadline and a card limit. | 82 | 10 | 54% |
| Household savings plan from records and familyHousehold | Ahead of time. Coming bills by month, debt order and two target months are checked against a cash‑flow model of the plan's own figures. | 51 | 8 | 85% |
| Course registration optimised for earliest graduationUniversity portal | Best option. One check rewards the same‑slot course that leads to graduating soonest, though keeping the course already held breaks no rule. | 14 | 15 | 93% |
| 12‑month equipment plan with simulated shop managerSmall business | Cost over time. A 12‑month profit model prices the plan, scaled by the share of key facts it quotes from the manager. | Cost model | 13 | 57% |
| Seasonal bulk order run with simulated staffSmall business | Cost over time. 70% of the reward is profit across the order's weeks, pricing production, overtime, agency hours, freezer space and an oven service. | 23 | 8 | 75% |
| Three‑year repair‑or‑replace plan with simulated managerSmall business | Cost over time. A 36‑month profit model prices lead times, warranties, energy and pans, scaled by the weighted share of key facts quoted. | Cost model | 12 | 93% |
| Smart‑home retrofit: purchase plan, wiring and local control softwareHousehold | Cost over time. The purchase plan must fit the budget and still buy a drive that holds 90 days of footage. | 121 | 7 | 92% |
| Month‑long parcel fleet dispatch, notes‑only memorySmall business | Ahead of time. Worn trucks need the single workshop bay before they break, and each day's promised windows face a pass mark. | 29 | 4 | 100% |
| Tea‑factory cash desk over thirteen weekly sessionsSmall business | Ahead of time. The bank limit moves with unpaid invoices, and passing it ends the business, so offers and auctions need a daily forecast. | 64 | 5 | 100% |
| Two‑site spares pick lists and stock plansManufacturing | Ahead of time. Stock plans must keep each site's downtime risk under the limit at weeks 4 and 8, within caps on orders and drawers. | 35 | 6 | 100% |
| Packing‑line project purchasing over twelve mail‑driven weeksManufacturing | Ahead of time. Items, some added by design revisions, must reach site the working day before their stage, and quoted lead times can mislead. | 14 | 6 | 100% |
| Two‑site spares buying through supplier part withdrawalsManufacturing | Ahead of time. Replacements must be ordered early enough for the real lead time, in enough for both sites, within a monthly budget. | 62 | 5 | 100% |
| Coffee roaster materials planning across two seasonsManufacturing | Ahead of time. Coffee and packaging must be bought ahead of seasonal demand and disruptions, within storage caps and shelf life. | 112–166 | 5 | 98% |
| Tea factory rooftop solar and battery design with its business caseEnergy engineering | Cost over time. Exports earn a fifth of the import price and credits lapse after the idle season, so the biggest system isn't the best. | 18 | 6 | 92% |
| Coffee web‑shop order desk through a four‑act Christmas peakSmall business | Ahead of time. Second shifts must be booked two working days ahead, and short coffees taken off sale before stock runs out. | 207–213 | 5 | 94% |
| Three‑day hospital ward care coordination, notes‑only memoryHospital | Ahead of time. Tests need their preconditions, fasting and transport arranged before booked slots, and doses must land inside their windows. | 290–415 | 6 | 100% |
| Server‑room cooling outage in a 3D building with model‑played staffFacilities | Ahead of time. With no cooling the servers shut down within minutes and each call takes several, so a stopgap has to come first. | 31 | 7 | 33% |
| Room furnishing from photos: measure, match look‑alikes, order and planFurnishing service | Best option. Three pieces arrive in time from only one shop, and each order adds a delivery fee, so the split decides the cost. | 28 | 8 | 80% |
| Flood‑claim replacement from dated photos: what stood, what stays, look‑alikes and set‑upInsurance replacement | Ahead of time. Each order takes one delivery date, and five pieces arrive inside the window at only one of their two shops. | 25 | 8 | 85% |
| Row‑house rooftop solar designed and quoted from plans, imagery and photosHome solar | Cost over time. Half the reward is the family's ten‑year gain, which a neighbour's planned extension and the family's appliance timers both move. | 15 | 6 | 99% |
Checks: a range where the count varies by attempt. Each environment's datasheet shows how its checks become the reward.
Try one of our three sample environments
Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.
AsapiThe AI Agent Trials