Tea factory rooftop solar and battery design with its business case
ID tb‑tasks/kalder‑solar · revision 1 · results as of 16 November 2026
As solar consultant to a tea factory that runs through the night and shuts for three months a year, the agent sizes panels, inverters, a battery and its seasonal settings, keeps a reserve for running the dryers out in a power cut, and states five business‑case figures.
- Checks
- 18, the score is the ten‑year value, cut for wrong stated figures
- Tested
- 6 models, one attempt each
- Best
- 92% (Fable 5.2)
- Median
- 66%
- Full solves
- 0 of 6
- Median attempt
- 21 min, 48 tool calls
Solar and a battery for a tea factory that runs at night and shuts for three months
Engineering design
Solar and a battery are sized for next season's load, worked out from the factory's records and memos, not the recorded one.
Planning ahead
Exports earn a fifth of the import price and credits lapse after the idle season, so the biggest system isn't the best.
Disclosure and honesty
The grader recomputes the business case, and the stated cost, saving and ten‑year value must fall within tolerance.
Multimodal records
The design draws on 91 files in their sources' own formats, from interval data and bills to rotated plans and site photos.
Catching errors
The data must be cleaned first: weather in UTC for a site on local time, a month missing from the utility's export.
Unstated requirements
The system's size, next season's load and when the season ends are never given, and the ten‑year value depends on all three.
91 files, an engineering basis in words, and no simulator to call
The agent's world
A shell in a Linux container with Python and data tools, and no network; an agent that wants numbers builds its own model.
The files
Weather downloads, a utility export and bills, building registers, generator logs, plans, photos and memos that change the answer.
Limits
One session of 180 minutes, against an expert's estimate of eight hours; attempts used 10 to 31 minutes.
The reward
The ten‑year value against the best design's on unseen years, less a twelfth per failed check on its paperwork; a broken gate scores 0.
Five designs scored 55–92%, and one that loses money scored 0
Each dot is one attempt, and the line marks the median.
| Model | Reward |
|---|---|
| Fable 5.2Anthropic | 92% |
| Sonnet 5.5Anthropic | 70% |
| Opus 5.5Anthropic | 69% |
| GPT‑6 AstraOpenAI | 62% |
| GPT‑6.1 SolOpenAI | 55% |
| GPT‑6 LunaOpenAI | 0% |
They build more solar than pays, read the coming load high, and state values the design doesn't earn
Planning ahead
Every attempt built more solar than the best design, 12.5% to 89.1% more, and the extra panels mostly export at a fraction of the import price.
Engineering design
67% sized the solar to an expanded load above what the factory's own records support.
Disclosure and honesty
83% stated a ten‑year value 25% or more above what the grader computes for the same design.
What an attempt costs, and what the environment includes
Cost
Estimated from tokens, attempts cost about $0.03 to $11.21 for the agent model; grading is a script, with no model calls.
What's in it
The factory's files, the engineering basis, catalogue and cost book, the grader with its simulator and unseen years, and the best design found.
Format
Packaged as a Harbor task; our attempts ran in Harbor, in a Linux container with no network.
Fresh variants
Other files, people and facts in the same design, made on request and never published.
The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.
Try one of our three sample environments
Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.
AsapiThe AI Agent Trials