Tea‑factory cash desk over thirteen weekly sessions
ID tb‑tasks/leafline‑cash‑desk · revision 3 · tested 1 and 13 October 2026
Each simulated Monday for 13 weeks, the agent prices and sizes its offers to the buyers who enquired, picks the auction lots and decides whether to ask the owner for money, keeping the factory inside a moving bank limit. It works through 16 independent scenarios, remembering only its own notes.
- Checks
- 64, 4 gates in each of 16 scenarios
- Tested
- 5 models, one attempt each
- Best
- 100% (Opus 5.5)
- Median
- 81%
- Full solves
- 1 of 5
- Median attempt
- 559 min, 3849 tool calls
A tea factory's cash desk, kept inside a moving bank limit for 13 weeks
Long‑horizon work
Thirteen weekly sessions keep only the agent's files, and buyers' prices have to be re‑learned from accepted and refused quotes each week.
Stateful tool use
Each week's sales move the books and bank balance the next week starts from, and the owner's money takes three weeks to arrive.
Planning ahead
The bank limit moves with unpaid invoices, and passing it ends the business, so offers and auctions need a daily forecast.
When to act or escalate
Asking the owner for money fails when the business could manage without it, and asking too late lets it go under.
A SQLite ledger, six buyers played by code and 16 scenarios
The agent's world
A shell in a container with Python and SQLite, one fresh session each Monday, and one desk command to see the week and submit.
The buyers
Six per scenario, played by code: each accepts a quote at or below a hidden price that drifts with the market, and refuses above it.
Scenarios
Sixteen, two of each of eight types, from a steady season to the largest credit buyer defaulting or news in the inbox.
The reward
The share of scenarios passing four gates: run to the end, inside the bank limit, no desk rule broken, no needless request.
One attempt passed all 16 scenarios; the rest passed 12% to 88%
Each dot is one attempt, and the line marks the median.
| Model | Reward |
|---|---|
| Opus 5.5Anthropic | 100% |
| GPT‑6.1 SolOpenAI | 88% |
| GPT‑6 AstraOpenAI | 81% |
| Sonnet 5.5Anthropic | 62% |
| GPT‑6 LunaOpenAI | 12% |
They ask the owner when they needn't, ask too late, and break rules on stale records
When to act or escalate
Three attempts asked the owner for money in scenarios that could get through without it, in four cases after their own weak selling made it look needed.
In the scenario where the biggest buyer stops paying, two attempts asked the owner a week too late or later, and the factory went under before the money arrived.
Stateful tool use
Four attempts broke a standing desk rule in 11 scenarios between them, eight of those GPT‑6 Luna's, mostly by matching payments by exact reference, so paid buyers looked overdue.
Planning ahead
The three that wrote a payment matcher and a daily cash forecast passed 13 or more scenarios; GPT‑6 Luna wrote neither, passed 2 and went under in 5.
What an attempt takes, and what the environment includes
An attempt
The median attempt used about 130 million tokens, cached input included.
What's in it
The 16 scenarios' books, documents and inboxes, the weekly simulator with its hidden scenario files, and the reference desk's decisions.
Format
Our attempts ran in Harbor, each scenario a multi‑step task of 13 weekly sessions; other packaging on request.
Fresh variants
Other files, people and facts in the same design, made on request and never published.
The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.
Try one of our three sample environments
Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.
AsapiThe AI Agent Trials