AsapiThe AI Agent Trials
Asapi / Catalogue / Month‑long parcel fleet dispatch, notes‑only memory
Environment datasheet · Small business

Month‑long parcel fleet dispatch, notes‑only memory

ID tb‑tasks/kestrel‑fleet‑dispatch · revision 1 · results as of 30 October 2026

As dispatcher of a simulated city parcel carrier for 28 days, the agent promises every receiver a two‑hour window, plans each day's trucks and answers breakdowns and delays through a watcher program and wake‑ups, remembering nothing between sessions but its own files.

Checks
29, one a day, plus a gate on the watcher's sandbox limits
Tested
4 models, one attempt each
Best
100% (GPT‑6.1 Sol and Opus 5.5)
Median
62%
Full solves
2 of 4
Median attempt
158 min, 716 tool calls
What it tests

A parcel fleet dispatched for 28 days, remembering only its own files

Long‑horizon work

Fresh sessions over a simulated month keep only the agent's files, and each day's promises and truck wear carry into the next.

Stateful tool use

Promises, truck wear and repairs carry over day to day, so each day's plan has to start from what's actually due.

Planning ahead

Worn trucks need the single workshop bay before they break, and each day's promised windows face a pass mark.

When to act or escalate

Wake‑ups are capped at four a day, so a watcher that fires on known or harmless events leaves later breakdowns unanswered.

How it works

A city simulator, 14 trucks, a watcher program and four wake‑ups a day

The agent's world

A shell in a container with one dispatch command, covering parcels, trucks, plans, the watcher and wake‑ups, to a simulator it can't inspect.

Sessions

A morning planning session each day and up to four wakes; the clock stops while the agent works, and only its files carry over.

Hidden state

True road conditions are never shown, a hub's delay notice is only an estimate, and trucks wear out on damaged roads.

The reward

One check a day, met when 96% of its parcels arrive on time; two missed days in a row end the trial.

Results

Two attempts met every day of the month; the other two were stopped on days 3 and 11

Each dot is one attempt, and the line marks the median.

MedianFully solved
4 modelsOne attempt each62%2 of 4
ModelReward
GPT‑6.1 SolOpenAI100%
Opus 5.5Anthropic100%
Sonnet 5.5Anthropic25%
GPT‑6 LunaOpenAI4%
What models get wrong

They drop promised parcels, repair trucks too late, and spend wakes on known problems

Long‑horizon work

  1. One planned its second day from new arrivals alone, leaving off parcels it had promised, then planned day 3 with no travel times and was stopped.

Planning ahead

  1. One booked worn trucks into the workshop only after drivers reported knocking, and had 8 breakdowns between days 5 and 11.

  2. One planned early trucks past the drivers' 10‑hour limit on a short‑staffed Sunday; they turned back with parcels aboard.

When to act or escalate

  1. From day 6, one spent its first wake each day on a truck already in repair or without a driver; on day 11 all four wakes were gone by 10:52, two later breakdowns went unanswered, and the trial ended.

Running it

What an attempt costs, and what the environment includes

Cost

Estimated from tokens, attempts cost about $0.22 to $30.39 for the agent model; no model runs inside the environment.

What's in it

The simulator and its scenario, the company documents and history, the grader, and a scripted expert with its watcher.

Format

Our attempts ran in Harbor as one multi‑step task, a session per morning or wake; other packaging on request.

Fresh variants

Other files, people and facts in the same design, made on request and never published.

The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.