Logistics & industry · Task 28 of 34
The parcel dispatcher
Dispatch a parcel fleet for a simulated month in fresh sessions, promising every customer a delivery window and keeping it through breakdowns, closures and a city known only from notes.
The task
Kestrel Parcel, a made-up company, delivers in a made-up city with a river, bridges, an expressway and a metro. The agent takes over from the previous dispatcher for a month: three hubs, 14 trucks, about 12,000 parcels. Each morning it promises every receiver a two-hour window and plans the trucks; during the day a watcher it wrote itself, and its own wake-ups, call it back for breakdowns, closures and late trucks. Every session is fresh, and only its notes carry over.
The prompt only asks it to promise windows and keep enough of them to meet each day's standard. The real test is running an operation over weeks: learning the city from the history and the drivers, noticing when the documents are wrong, servicing trucks before they fail, and keeping some attention for the afternoon.
What it tests
Judging over the long run
Servicing, the calendar and the use of wake-ups pay off days or weeks later.
Weighing sources that disagree
The directory, the maps, the history and the drivers do not always say the same thing.
Working out what the job needs
Holidays, hub timings and drivers' hours shape the plan without anyone pointing them out.
Learning from its own mistakes
What went wrong yesterday is only in its logs and notes.
Challenges
1No memory
Every session starts fresh, so what the city has taught the agent lives only in its notes.
2A directory that is not always right
A business's real hours can differ from what the directory says, as in any real address book.
3Trucks wear out
Trucks that are never serviced break down later in the month.
4Calendar effects
Holidays and events change the fleet and the roads.
5Limited wake-ups
Spending them early leaves nothing for an afternoon breakdown.
Results
| Model | Standard | Time (standard) | Cost (standard) |
|---|---|---|---|
| Opus 5.5 | 100% | 186 min | $30.39 |
| Sonnet 5.5 | 25% | 130 min | $12.66 |
One run per model and setup. 100% = everything right (or, for a plan scored in money, the best plan); 0% = the obvious answer or worse.
Setups: Standard, the job as asked.
What models got right and wrong
- Two models ran this task, both in Claude Code. Opus 5.5 met all 28 days (100.0%), missing 19 of 12,239 parcels.
- Opus 5.5 spent its untimed first session reading every document and the whole history, built its own planning tools, and sent its most worn truck to the workshop each morning, so no truck broke down all month.
- Sonnet 5.5 (25.0%) met 7 days and failed on the eleventh, its second missed day in a row. It learned each lesson one miss late: it planned early trucks past the drivers' 10-hour limit on a short-staffed Sunday, and twice a late hub hit parcels it had already promised for the earliest window.
- Sonnet 5.5 booked the workshop for some warned trucks but had eight breakdowns in eleven days. On its last two days its first wake-up went on a truck already in repair, and a breakdown came after its last wake-up was used.
- There is no hinted or told-everything setup for this task: every score is on the job as asked.
Test environment
- Kind
- Simulated city logistics: dispatching the fleet of Kestrel Parcel, a made-up parcel company in a made-up city with a river, bridges, an expressway and a metro, for 28 simulated days: three hubs, 14 trucks, about 12,000 parcels. A deterministic simulator in the container, which the agent cannot read, runs the city
- Agent's interface
- A terminal in a fresh Docker container (Ubuntu 24.04, Python 3; 4 CPUs, 8 GB), one fresh session per morning or wake as an unprivileged user, driven by the agent's own command-line tool through harbor: Claude Code
- Documents
- 17 company documents in
/docs(12 Markdown: a README, the instruction, the service policy, the fleet, the calendar, city notices, the city and metro maps, the previous ops lead's handover notes, a data guide, the tool guide, the watcher's interface; 5 CSV: roads, nodes, zones, the receivers' directory, an old travel-time matrix) and eight weeks of operations history in/data/days: 56 day folders of 16 files each (7 CSV, 4 JSON, 3 Markdown, 1 JSONL, 1 gzipped CSV of GPS traces) and an index CSV; 914 files in all. Each night the day just played is added - Applications
- One command,
fleet: parcels, receivers, trucks, promises, the inbox, plans (validate, submit, show), the watcher, wake-ups, the drivers' morning debrief and a traffic one-pager; during the day, truck status, reroutes, transfers, recovery of a failed truck's load, the workshop, recalls, standby vans and route hints. Every command can print JSON. No other apps - Web
- Not needed: nothing the task asks about is online. The container can reach the internet. Claude Code's web tools were on in both runs (WebFetch for Opus 5.5 on Bedrock; WebFetch and WebSearch for Sonnet 5.5 on the Claude plan), and neither run used them
- People
- None who answer: receivers do not reply to the windows promised them. Drivers speak only through the simulator's code: messages during the day, notes in the history, and a morning debrief whose words a language model wrote when the task was built, chosen by the simulator and kept nowhere
- Time
- A simulated clock from Monday 5 October 2026, for 28 days: a morning session each day at 06:00, before the trucks load, and up to 4 wake sessions a day, called by the agent's watcher or its own wake-ups. The clock stops while the agent works
- Actions
- Promise each parcel a day and a 2-hour window; submit each day's plan (trucks, stops, route hints) and change it during the day; transfer parcels between trucks, recover a failed truck's load, recall a truck, call out a standby van, book the workshop; install a watcher; set wake-ups; hand in a traffic one-pager
- Deliverable
- No single file: each day's promises and JSON plan, handed in through
fleet, plus a watcher (watcher.py) and wake-ups