AsapiThe AI Agent Trials
Asapi / Capabilities / Planning ahead
Capability · 21 environments

Planning ahead

Many choices only show their cost or value weeks or months later. Doing well means counting the cost over time, preparing ahead, and choosing by where an option leads, not how it looks today.

What it takes

Three things an agent has to get right

Count costs and value over time

In several, a profit model prices each plan over months or years, so a choice that looks good now can lose money by the end.

Prepare ahead

Lead times, booking windows and seasonal peaks mean orders, shifts and tests have to be arranged before they're needed.

Choose by where it leads

The best option is often not the obvious one: a repair over a replacement, a smaller solar array, a course that opens up later terms.

What models get wrong

They misjudge later value, start too late, or take the obvious option

Value misjudged

  1. Tea factory rooftop solar: every attempt built a bigger solar array than pays off, up to nearly twice the size of the best design; most of the extra power sells to the grid at a fifth of the import price.

  2. Household savings plan: 63% of attempts built the plan on uncertain weekend side income, with no version that works without it.

Not ready in time

  1. Tea‑factory cash desk: in 40% of attempts, the agent asked the owner for money too late in one of the 16 scenarios, since money takes three weeks to arrive, and the business went under.

  2. Coffee roaster materials planning: in 86% of attempts, the agent failed in at least one season to prepare for a large customer order that the commercial manager had warned it about.

The obvious choice

  1. 12‑month equipment plan: 69% of attempts bought a new cabinet when a cheaper repair would fix the old one.

  2. Course registration optimised for earliest graduation: 93% of attempts kept the course already held in a contested slot and missed the one that would let the student graduate sooner.

Results

Partial credit for most attempts, and full solves in six of the 21 environments

Each dot is one attempt at the whole job, and the line marks the median. Larger dots are full solves, scoring 100 out of 100.

EnvironmentMedianFully solved
Cancelled‑flight family rebooking against a ticking clock8 models × 3 attempts70%0 of 24Household solar purchase plan with simulated contacts10 models24%0 of 10Household savings plan from records and family8 models50%0 of 8Course registration optimised for earliest graduation15 models79%0 of 1512‑month equipment plan with simulated shop manager13 models0%0 of 13Seasonal bulk order run with simulated staff8 models46%0 of 8Three‑year repair‑or‑replace plan with simulated manager12 models0%0 of 12Smart‑home retrofit: purchase plan, wiring and local control software7 models76%0 of 7Month‑long parcel fleet dispatch, notes‑only memory4 models62%2 of 4Tea‑factory cash desk over thirteen weekly sessions5 models81%1 of 5Two‑site spares pick lists and stock plans6 models90%1 of 6Packing‑line project purchasing over twelve mail‑driven weeks6 models × 3 attempts100%13 of 18Two‑site spares buying through supplier part withdrawals5 models97%2 of 5Coffee roaster materials planning across two seasons5 models, 1 or 2 attempts each90%0 of 7Tea factory rooftop solar and battery design with its business case6 models66%0 of 6Coffee web‑shop order desk through a four‑act Christmas peak5 models90%0 of 5Three‑day hospital ward care coordination, notes‑only memory6 models95%1 of 6Server‑room cooling outage in a 3D building with model‑played staff7 models5%0 of 7Room furnishing from photos: measure, match look‑alikes, order and plan8 models × 3 attempts63%0 of 24Flood‑claim replacement from dated photos: what stood, what stays, look‑alikes and set‑up8 models × 3 attempts75%0 of 24Row‑house rooftop solar designed and quoted from plans, imagery and photos6 models71%0 of 6

Hover a dot for its model and reward.

The 21 environments

How each environment tests it

EnvironmentHow it tests thisChecksModelsBest
Cancelled‑flight family rebooking against a ticking clockTravel conciergeAhead of time. The clock moves with every page, form and message, and alternative seats sell out at fixed times, so securing seats early pays.61–63885%
Household solar purchase plan with simulated contactsHouseholdCost over time. Paying routes, from cash to finance, card instalments or waiting, are scored over time against a rebate deadline and a card limit.821054%
Household savings plan from records and familyHouseholdAhead of time. Coming bills by month, debt order and two target months are checked against a cash‑flow model of the plan's own figures.51885%
Course registration optimised for earliest graduationUniversity portalBest option. One check rewards the same‑slot course that leads to graduating soonest, though keeping the course already held breaks no rule.141593%
12‑month equipment plan with simulated shop managerSmall businessCost over time. A 12‑month profit model prices the plan, scaled by the share of key facts it quotes from the manager.Cost model1357%
Seasonal bulk order run with simulated staffSmall businessCost over time. 70% of the reward is profit across the order's weeks, pricing production, overtime, agency hours, freezer space and an oven service.23875%
Three‑year repair‑or‑replace plan with simulated managerSmall businessCost over time. A 36‑month profit model prices lead times, warranties, energy and pans, scaled by the weighted share of key facts quoted.Cost model1293%
Smart‑home retrofit: purchase plan, wiring and local control softwareHouseholdCost over time. The purchase plan must fit the budget and still buy a drive that holds 90 days of footage.121792%
Month‑long parcel fleet dispatch, notes‑only memorySmall businessAhead of time. Worn trucks need the single workshop bay before they break, and each day's promised windows face a pass mark.294100%
Tea‑factory cash desk over thirteen weekly sessionsSmall businessAhead of time. The bank limit moves with unpaid invoices, and passing it ends the business, so offers and auctions need a daily forecast.645100%
Two‑site spares pick lists and stock plansManufacturingAhead of time. Stock plans must keep each site's downtime risk under the limit at weeks 4 and 8, within caps on orders and drawers.356100%
Packing‑line project purchasing over twelve mail‑driven weeksManufacturingAhead of time. Items, some added by design revisions, must reach site the working day before their stage, and quoted lead times can mislead.146100%
Two‑site spares buying through supplier part withdrawalsManufacturingAhead of time. Replacements must be ordered early enough for the real lead time, in enough for both sites, within a monthly budget.625100%
Coffee roaster materials planning across two seasonsManufacturingAhead of time. Coffee and packaging must be bought ahead of seasonal demand and disruptions, within storage caps and shelf life.112–166598%
Tea factory rooftop solar and battery design with its business caseEnergy engineeringCost over time. Exports earn a fifth of the import price and credits lapse after the idle season, so the biggest system isn't the best.18692%
Coffee web‑shop order desk through a four‑act Christmas peakSmall businessAhead of time. Second shifts must be booked two working days ahead, and short coffees taken off sale before stock runs out.207–213594%
Three‑day hospital ward care coordination, notes‑only memoryHospitalAhead of time. Tests need their preconditions, fasting and transport arranged before booked slots, and doses must land inside their windows.290–4156100%
Server‑room cooling outage in a 3D building with model‑played staffFacilitiesAhead of time. With no cooling the servers shut down within minutes and each call takes several, so a stopgap has to come first.31733%
Room furnishing from photos: measure, match look‑alikes, order and planFurnishing serviceBest option. Three pieces arrive in time from only one shop, and each order adds a delivery fee, so the split decides the cost.28880%
Flood‑claim replacement from dated photos: what stood, what stays, look‑alikes and set‑upInsurance replacementAhead of time. Each order takes one delivery date, and five pieces arrive inside the window at only one of their two shops.25885%
Row‑house rooftop solar designed and quoted from plans, imagery and photosHome solarCost over time. Half the reward is the family's ten‑year gain, which a neighbour's planned extension and the family's appliance timers both move.15699%

Checks: a range where the count varies by attempt. Each environment's datasheet shows how its checks become the reward.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.