AsapiThe AI Agent Trials
Asapi / Catalogue / Coffee roaster materials planning across two seasons
Environment datasheet · Manufacturing

Coffee roaster materials planning across two seasons

ID tb‑tasks/thornbury‑supply · revision 3 · tested 14 to 28 October 2026 · regraded 18 November 2026

As materials planner at a coffee roaster with three contract customers, the agent buys green coffee and packaging and files a weekly roast‑and‑pack plan and a monthly pricing advisory, through a summer and an autumn window of 16 weeks each. A customer let down too often ends the window.

Checks
112–166, 166 when both seasons run to the end
Tested
5 models, 1 or 2 attempts each
Best attempt
98% (GPT‑6.1 Sol)
Median
90%
Full solves
0 of 7
Median attempt
113 min
What it tests

Coffee and packaging planned through a summer and an autumn of 16 weeks

Long‑horizon work

Each 16‑week season runs up to 80 fresh sessions keeping only the agent's files, and a customer let down too often ends it early.

Stateful tool use

Stock, orders and holds carry over week to week in each season, and a material the agent stops tracking can run dry.

Planning ahead

Coffee and packaging must be bought ahead of seasonal demand and disruptions, within storage caps and shelf life.

When to act or escalate

Preparing for the sales manager's real tip is rewarded, while buying on a tip that comes to nothing or a withdrawn warning fails.

How it works

A plant simulator, three customers, five supplier desks and two seasons

The agent's world

One planning tool to a simulator: orders and holds, a weekly roast‑and‑pack plan, a monthly pricing advisory and supplier mail.

The customers

Three, with weekly call‑offs; one ends its agreement after repeated short or late deliveries, and the season ends there.

Events

Each season brings disruptions, a sales tip that becomes an order, one that doesn't, a forecast that runs high and a withdrawn warning.

The reward

The mean of the two seasons' scores, each the share of weighted checks passed, from deliveries and pricing to restraint.

Results

Four attempts scored 90–98%, and three fell to 65% or less

Each dot is one attempt, and the line marks the median.

MedianFully solved
5 models1 or 2 attempts each90%0 of 7
ModelMean reward
GPT‑6.1 SolOpenAI98%
GPT‑6 AstraOpenAI95%
Opus 5.5Anthropic91%
Sonnet 5.5Anthropic50%
GPT‑6 LunaOpenAI27%

GPT‑6.1 Sol's and GPT‑6 Astra's attempts also passed both seasons' gates.

What models get wrong

They leave the real tip uncovered, let materials run out, and misprice a closure

When to act or escalate

  1. 69% of the seasons that reached it neither bought nor held coffee for the tip that became an order, and the coffee sold out.

Stateful tool use

  1. All four of Sonnet 5.5's seasons ended early when something it had stopped tracking ran out.

  2. GPT‑6 Luna sized plans from a count of finished goods that still held cases that week's despatch would take, so lines shipped short.

Planning ahead

  1. Both of Opus 5.5's autumns priced products below their true break‑even, counting coffee the port closure held back as due.

Running it

What an attempt costs, and what the environment includes

Cost

Estimated from tokens, attempts cost about $0.37 to $107.32 for the agent model; nothing else in the environment calls a model.

What's in it

Both seasons' simulators with the plant's and a sister plant's records, the scripted customers and suppliers, the grader, and a reference solution.

Format

Packaged as a Harbor task on request, one per season; Codex's attempts ran in Harbor, Claude Code's through a local driver.

Fresh variants

Other files, people and facts in the same design, made on request and never published.

The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.