AsapiThe AI Agent Trials
Asapi / Catalogue / Two‑site spares pick lists and stock plans
Environment datasheet · Manufacturing

Two‑site spares pick lists and stock plans

ID tb‑tasks/spares‑crib · revision 1 · results as of 16 November 2026

As stores controller for a coffee roaster's two sites, the agent keeps the technicians' list of parts for each repair, learned from a year of work logs, and a stock plan that keeps downtime risk under the company's limit. One session a week; only a workspace carries over.

Checks
35, 33 repair reports and 2 stock checks
Tested
6 models, one attempt each
Best
100% (Opus 5.5)
Median
90%
Full solves
1 of 6
Median attempt
22 min, 124 tool calls
What it tests

Two sites' pick lists and stock, learned from a year of work logs

Long‑horizon work

Nine fresh sessions keep only the agent's files, and the stock plan, refiled week by week, is checked at weeks 4 and 8.

Stateful tool use

Stock moves as technicians take parts, and the stock plan is tested twice against the downtime limit at both sites.

Planning ahead

Stock plans must keep each site's downtime risk under the limit at weeks 4 and 8, within caps on orders and drawers.

Unstated requirements

Which parts a repair really takes shows only in the work logs; the documented lists are right on 12 of 33 reports.

How it works

Two sites' records in two formats, scripted technicians and nine sessions

The agent's world

A terminal in a container with one tool to a simulator: file the pick list and stock plan, count drawers, end the session.

The records

One site's scanned paper log as OCR text, with errors, and the other's database export; the two number parts differently.

The technicians

Played by code, they take the list's entry to the stores, reject what doesn't suit, use what the job needs and return the rest.

The reward

The pick list's accuracy on 33 repair reports, measured up from the manuals' 12 right, times the credit from two stock checks.

Results

Three attempts scored 95% or more, and the two that failed a stock check 62% and 74%

Each dot is one attempt, and the line marks the median.

MedianFully solved
6 modelsOne attempt each90%1 of 6
ModelReward
Opus 5.5Anthropic100%
Fable 5.1Anthropic95%
Sonnet 5.5Anthropic95%
GPT‑6 LunaOpenAI86%
GPT‑6 AstraOpenAI74%
GPT‑6.1 SolOpenAI62%
What models get wrong

They count the wrong part as cover, miss a new combined repair, and follow one‑offs

Stateful tool use

  1. Two counted a shatterproof lamp as cover for the ordinary one; by week 8 neither site held an ordinary lamp, and the stock check failed.

Unstated requirements

  1. 67% picked the wrong parts for a two‑bearing failure at least once, though the work logs show it takes a shaft kit.

  2. One dropped a belt's pins after a single job waited for them, and listed a manual's bearing it had seen technicians reject.

Running it

What an attempt costs, and what the environment includes

Cost

Estimated from tokens, attempts cost about $0.11 to $12.33 for the agent model; nothing else in the environment calls a model.

What's in it

The simulator with both sites' records, the technicians' rules and failure schedule, the grader with its replay check, and a reference solution.

Format

Our attempts ran on its own multi‑step Harbor build, one step per session; other packaging on request.

Fresh variants

Other files, people and facts in the same design, made on request and never published.

The same plant's buying environment keeps the shelves stocked as suppliers withdraw parts. The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.