Long‑horizon operation with notes‑only memory

Fresh sessions, and only the agent's files in between

Span

From 72 simulated hours on a hospital ward to two 16‑week windows at a coffee roaster. The cash desk has 13 weekly sessions in each of 16 scenarios; the ward used 127 to 217 wakes; the order desk up to 48 sessions in each of four acts.

What carries over

Only the agent's workspace or notes folder is meant to carry over. Each session starts the harness afresh, with no earlier conversation; between sessions a verifier stops leftover processes and deletes the harnesses' memory folders, except in Claude Code's attempts at the coffee roaster's materials planning.

Notes under limits

In the tea‑factory ops chat, the agent reads each scenario's 17 to 20 days of chat once and keeps at most 16 KB of notes, no more than a fifth of lines copied. The history is then removed, and each of 106 check points starts from the notes as saved.

Heard once

The fleet's drivers give their debriefs in the morning session only, and they are never archived: what isn't written down that morning is gone. They aren't yet required: a scripted agent that never hears them still passes the month.

What opens a session

Mail, at the end of each working day that brings some, on the packing line, in spares buying and in the materials planning; each working day and each reply at the order desk; a fixed week at the cash desk and the spares stores.

On a schedule

The journey planner's three releases fall on day 0 and after weeks 1 and 4 of five field weeks; the tea‑factory chat has four check points a working day.

Waking itself

On the ward the agent sets its own wake‑ups, at most 207, and new orders, staff messages and most page replies also wake it. In the fleet it writes a watcher program, called every simulated minute in a sandbox; at most four wakes a day count.

Late consequences

Truck wear builds over days on damaged roads before a breakdown; the owner's money reaches the cash desk weeks after the request; some suppliers' real lead times differ from their quotes, as the purchasing records show; a rider let down stops using the journey planner.

Same world each time

Nine are simulators and the tea‑factory chat serves messages written in advance; people are scripted, with no language model inside, so the same commands give the same weeks. Six check after the last session that replaying the logged commands in a fresh simulator matches the attempt's trace.

Scoring

Rewards count outcomes over the whole span: days on time out of 28, scenarios passing four gates out of 16, ward obligations met, repairs that didn't wait, items on site before their stage, week‑five requests planned right. Code decides all of it; no LLM judge.

Stop rules

Two missed days in a row end the fleet's month; going under ends a cash‑desk scenario; a customer let down too often ends a roaster window; runaway service costs end an order‑desk act. In the spares environments and the packing line, idle attempts stop after week 4 or 6.

Routine lapses, paid for days or weeks later

Each count covers every model tested on the environment, with only the request; where a model made several attempts, the line says so.

1. Repairs after the fault. Month‑long parcel fleet dispatch: 1 of 2 attempts, one per model, booked worn trucks into the one workshop bay only after drivers reported knocking: 8 breakdowns between days 5 and 11. The other booked trucks in early, from the history, and had none all month.

2. Handoffs that drift. Three‑day hospital ward: 2 of 6 attempts got all seven shift handoffs exactly right; the others got 6, 6, 3 and 1. The 99% attempt listed a finished one‑night dose as still due, after writing its own reading of the order into its notes.

3. Its own record ignored. Packing‑line project purchasing: in 5 of 18 attempts (6 models × 3), two of Sonnet 5.5's and all of GPT‑6 Luna's, an order from the cheaper seller on its short quote, though its record showed weeks longer, left an item late or bought twice.

4. Stopped tracking. Coffee roaster materials planning: all 4 of Sonnet 5.5's windows (2 attempts) ended early when something it had stopped tracking ran out, and both of GPT‑6 Luna's, on plans counting stock already called off. Opus 5.5's, GPT‑6.1 Sol's and GPT‑6 Astra's ran to the end.

5. Half a repair restocked. Two‑site spares buying: 2 of 5 attempts, one per model, reordered one part of a two‑part repair when its bin emptied and missed that the other was down to one, which no mail flags: 4 repairs waited. The other 3 reordered that part in time.

6. A mistake left running. Coffee web‑shop order desk: 1 of 5 attempts left a mistake in its scheduled job that no later session checked against the records, and finance took the desk back partway through act 2.

Full solves in six of the ten, most of them on the packing line

Each dot is one attempt with only the request; the line marks the median. Larger dots are full solves.

Hover a dot for its model and reward.

How each one spreads the job over sessions

Checks: a range where the count varies by attempt. Best, told the rules: a dash, since each of these has one version of the request.

What we haven't measured

• Doing nothing doesn't score 0 everywhere: posting nothing scores 50% in the tea‑factory ops chat, an idle agent earns 39% in the spares buying and 18% in the spares pick lists, and 48% to 53% an act at the order desk, though their graders fail it.
• The parcel fleet has results from two Anthropic models only. Repeat attempts exist only on the packing line and at the coffee roaster's materials planning, so how much attempts vary is mostly unmeasured.
• Notes‑only memory holds as far as each cleanup reaches: a ward file outside the notes folder would persist, and four tea‑chat attempts' scratch folders weren't cleared. Claude Code's attempts at the materials planning ran outside a container, beside the simulator and grader, without the cleanup; transcripts were checked afterwards.
• Every attempt in an environment meets the same scenarios, with the same scheduled events on the same days, so these results don't show how an agent copes with a different sequence.
• Our results don't include the same jobs with memory kept between sessions, so they don't show how much the notes‑only design costs.