AsapiThe AI Agent Trials
Asapi / Catalogue / Tea factory rooftop solar and battery design with its business case
Environment datasheet · Energy engineering

Tea factory rooftop solar and battery design with its business case

ID tb‑tasks/kalder‑solar · revision 1 · results as of 16 November 2026

As solar consultant to a tea factory that runs through the night and shuts for three months a year, the agent sizes panels, inverters, a battery and its seasonal settings, keeps a reserve for running the dryers out in a power cut, and states five business‑case figures.

Checks
18, the score is the ten‑year value, cut for wrong stated figures
Tested
6 models, one attempt each
Best
92% (Fable 5.2)
Median
66%
Full solves
0 of 6
Median attempt
21 min, 48 tool calls
What it tests

Solar and a battery for a tea factory that runs at night and shuts for three months

Engineering design

Solar and a battery are sized for next season's load, worked out from the factory's records and memos, not the recorded one.

Planning ahead

Exports earn a fifth of the import price and credits lapse after the idle season, so the biggest system isn't the best.

Disclosure and honesty

The grader recomputes the business case, and the stated cost, saving and ten‑year value must fall within tolerance.

Multimodal records

The design draws on 91 files in their sources' own formats, from interval data and bills to rotated plans and site photos.

Catching errors

The data must be cleaned first: weather in UTC for a site on local time, a month missing from the utility's export.

Unstated requirements

The system's size, next season's load and when the season ends are never given, and the ten‑year value depends on all three.

How it works

91 files, an engineering basis in words, and no simulator to call

The agent's world

A shell in a Linux container with Python and data tools, and no network; an agent that wants numbers builds its own model.

The files

Weather downloads, a utility export and bills, building registers, generator logs, plans, photos and memos that change the answer.

Limits

One session of 180 minutes, against an expert's estimate of eight hours; attempts used 10 to 31 minutes.

The reward

The ten‑year value against the best design's on unseen years, less a twelfth per failed check on its paperwork; a broken gate scores 0.

Results

Five designs scored 55–92%, and one that loses money scored 0

Each dot is one attempt, and the line marks the median.

MedianFully solved
6 modelsOne attempt each66%0 of 6
ModelReward
Fable 5.2Anthropic92%
Sonnet 5.5Anthropic70%
Opus 5.5Anthropic69%
GPT‑6 AstraOpenAI62%
GPT‑6.1 SolOpenAI55%
GPT‑6 LunaOpenAI0%
What models get wrong

They build more solar than pays, read the coming load high, and state values the design doesn't earn

Planning ahead

  1. Every attempt built more solar than the best design, 12.5% to 89.1% more, and the extra panels mostly export at a fraction of the import price.

Engineering design

  1. 67% sized the solar to an expanded load above what the factory's own records support.

Disclosure and honesty

  1. 83% stated a ten‑year value 25% or more above what the grader computes for the same design.

Running it

What an attempt costs, and what the environment includes

Cost

Estimated from tokens, attempts cost about $0.03 to $11.21 for the agent model; grading is a script, with no model calls.

What's in it

The factory's files, the engineering basis, catalogue and cost book, the grader with its simulator and unseen years, and the best design found.

Format

Packaged as a Harbor task; our attempts ran in Harbor, in a Linux container with no network.

Fresh variants

Other files, people and facts in the same design, made on request and never published.

The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.