AsapiThe AI Agent Trials
Asapi / Catalogue / Two‑site spares buying through supplier part withdrawals
Environment datasheet · Manufacturing

Two‑site spares buying through supplier part withdrawals

ID tb‑tasks/spares‑sourcing · revision 1 · tested 13–15 October 2026 · regraded 18 November 2026

As buyer for a coffee roaster's two spare‑parts stores, the agent keeps every repair's parts on the shelf while suppliers withdraw six of them: it picks each replacement, gets it through change control and orders in time for both sites. Only a workspace carries over between sessions.

Checks
62, 61 repairs and a waste check that now costs points
Tested
5 models, one attempt each
Best
100% (GPT‑6 Astra and GPT‑6.1 Sol)
Median
97%
Full solves
2 of 5
Median attempt
61 min, 410 tool calls
What it tests

Spare parts kept on the shelf as suppliers withdraw six of them

Long‑horizon work

Dozens of sessions over twelve weeks keep only the agent's files, with several supplier threads open at once.

Stateful tool use

Suppliers withdraw parts, and replacements must be in stock at both sites before a repair waits; orders can be refused unexplained.

Planning ahead

Replacements must be ordered early enough for the real lead time, in enough for both sites, within a monthly budget.

How it works

Two sites' records, seven supplier desks and change control

The agent's world

A terminal in a container with one tool to a simulator: file bills of materials, mail suppliers, order, count drawers.

Withdrawals

Six parts are withdrawn on fixed days; in five of six brochures, the supplier's own suggested replacement breaks a site limit.

Change control

A lead technician answers a bill of materials in two working days, and one objected line rejects it.

The reward

Each repair waiting over a shift, and each wasted order beyond one, costs 1/37 of the reward; doing nothing scores 0.

Results

Two full solves, and every attempt scored 84% or more

Each dot is one attempt, and the line marks the median.

MedianFully solved
5 modelsOne attempt each97%2 of 5
ModelReward
GPT‑6 AstraOpenAI100%
GPT‑6.1 SolOpenAI100%
Sonnet 5.5Anthropic97%
Opus 5.5Anthropic95%
GPT‑6 LunaOpenAI84%
What models get wrong

They stock one site of two, miss the pins a repair also takes, and trust a cancelled order

Stateful tool use

  1. Two first filed replacement kits only for the site whose records name parts in words, not numbers: 3 repairs waited.

Planning ahead

  1. Two reordered drive belts when a bin emptied and missed that the pins each repair also takes were down to one.

Long‑horizon work

  1. GPT‑6 Luna relied on an order its supplier had written to cancel; separately, it left a site with fewer belts than one repair takes.

Running it

What an attempt takes, and what the environment includes

An attempt

The median attempt used about 15 million tokens, cached input included.

What's in it

The simulator with both sites' records, the suppliers, withdrawals and scripted people, the grader with its replay check, and a reference solution.

Format

Our attempts ran on its own multi‑step Harbor build, one step per session; other packaging on request.

Fresh variants

Other files, people and facts in the same design, made on request and never published.

It's the same plant as the pick lists and stock plans environment. The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.