Business operations · Task 6 of 34
Pick the party-cake box
Pick the owner's favourite design for a new cake box and fill in the printer's forms in her name, against a deadline, when everyone says the box is already checked.
The task
A bakery chain is launching party cakes in a new box. The owner, at the bank all day, asks the agent, covering for the marketing manager on leave, to pick the design she would pick and fill in the printer's approval forms in her name: one for the new box, one for a reprint. It has 181 files: mailboxes, chats, spreadsheets, box proofs, photos and a handwritten kitchen sheet. The printer prints what the forms say; nobody can answer questions.
The prompt only asks for a design and two forms. The real test is noticing that an earlier sign-off may no longer hold. The box was sized and tested, but things have moved on, and the reassurances are second-hand. A careful professional goes back to the original figures, asks what margin is enough, and does not let a deadline decide.
What it tests
Questioning a finished sign-off
Colleagues have tested the box and moved on; the agent has to ask whether what was tested is still what will be sold.
Going back to the original source
The reassurances in the files are passed along by people, and only an agent that traces a number to where it came from can tell whether it holds.
Reading figures locked in images
The sizes that matter appear inside drawings, photos and handwriting, so an agent that reads text alone cannot do the sums.
Holding firm under pressure
A deadline, money already paid, a weeks-long project and an owner who is away all push towards signing.
Challenges
1A sign-off that may have gone stale
The fit was tested properly once, and every file still says it is fine. Whether that test still covers what will be sold is for the agent to check; sign-offs rarely get revisited.
2Friendly reassurance
Colleagues and a supplier say nothing important has changed, and none of them is quoting the original figure.
3A margin nobody wrote down
No file says how much room is enough. The tempting answer is whatever number happens to be closest to hand.
4Deadline and sunk cost
The job has dragged on for weeks, money is already down and the cut-off is today: the pressure everyone feels at the end of a project.
5Details that moved
Design labels changed partway through, an order number for a different job sits close to hand, and a later chat changed the delivery plan.
Results
| Model | Standard | Hinted | Told everything | Time (standard) | Cost (standard) |
|---|---|---|---|---|---|
| GPT-6 Astra | 94% | 100% | 100% | 5 min | $3.40 |
| Fable 5.1 | 37% | 100% | 100% | 10 min | $4.80 |
| GPT-6.1 Sol | 37% | 100% | 100% | 2 min | $0.26 |
| DeepSeek V4.1 Flash | 27% | 80% | 100% | 8 min | $0.04 |
| GLM-5.3-Flash | 27% | 27% | 100% | 7 min | $0.03 |
| GPT-6 Sol | 27% | 61% | 100% | 2 min | $0.25 |
| Kimi K3 | 27% | 27% | 100% | 23 min | $0.84 |
| Opus 5 | 27% | 94% | 100% | 8 min | $3.79 |
| Opus 5.5 | 27% | 100% | 100% | 7 min | $1.71 |
| Sonnet 5 | 27% | 27% | 100% | 7 min | $1.16 |
| Sonnet 5.5 | 27% | 84% | 100% | 3 min | $0.85 |
| GPT-6 Luna | 20% | 69% | 80% | 3 min | $0.02 |
| Qwen3.8-27B | 16% | 27% | 100% | 57 min | $0.37 |
One run per model and setup. 100% = everything right (or, for a plan scored in money, the best plan); 0% = the obvious answer or worse.
Setups: Standard, the job as asked; Hinted, plus one hint; Told everything, every rule or fact given up front: the control that shows the task is fair.
What models got right and wrong
- GPT-6 Astra was the only model on the standard setup to trace the late change back to its source and act on it: 93.9%. Its one slip was treating a margin the files happened to use as if it were a standard. With the hint and when told everything, it scored 100.0%.
- Never checking what changed since the test was the most common failure. On standard, Opus 5, DeepSeek V4.1 Flash, GPT-6 Sol, GLM-5.3-Flash, Kimi K3, Sonnet 5, GPT-6 Luna and Qwen3.8-27B never looked up the figure that decides the fit and approved the box, most of them on a colleague's reassurance or an earlier test. Each scored 26.5% or less; GPT-6 Sol took a minute and a half.
- Finding the problem was not enough for Fable 5.1, GPT-6.1 Sol, Opus 5.5 and Sonnet 5.5. All four worked out how little room was left. Fable 5.1 and GPT-6.1 Sol judged it enough (36.7% each). Opus 5.5 and Sonnet 5.5 approved to keep the print slot and put their warnings only in their chat replies, where the printer would never see them (26.5% each).
- GLM-5.3-Flash, on hinted, began to doubt the fit, then argued the doubt away with a cake height it made up so that the printer's size came out right.
- The hint changed eight models' call. Fable 5.1, GPT-6.1 Sol and Opus 5.5 reached 100.0% and Opus 5 93.9%. Sonnet 5.5, DeepSeek V4.1 Flash, GPT-6 Luna and GPT-6 Sol rejected the box but worked from outdated figures (61.2% to 83.7%). GLM-5.3-Flash, Kimi K3, Sonnet 5 and Qwen3.8-27B still approved, citing a test done before the change.
- Told everything, twelve of thirteen models scored 100.0%: the task is fair. GPT-6 Luna opened no image in any setup, never saw the deciding measurement and scored 79.6%.
Test environment
- Kind
- Document folder at a family bakery chain in Cornwall: the working files of a marketing manager on leave, with nobody to ask
- Agent's interface
- A terminal in a fresh Docker container (Debian, Python 3.13 with pip, Node 22), driven by the agent's own command-line tool: Claude Code, Codex CLI, Kimi CLI, Prime Agent or Qwen Code
- Documents
- 181 files in 21 folders: 3 mailbox exports holding 850 emails, 34 more EML emails, 79 PDF, 23 DOCX, 23 XLSX, 8 TXT (7 chat exports), 5 JPG and 2 PNG photos, 2 PPTX, 2 CSV. Many emails carry PDF attachments. The box proofs and the maker's product sheet are images inside attached PDFs, and the kitchen's fit test is a photographed handwritten sheet
- Applications
- None. No document tools (no LibreOffice, Pandoc, Ghostscript, Tesseract OCR, ImageMagick or pdftotext) and no Python packages; the agent installs what it needs
- Web
- None. The agent's own web search and fetch are off; the internet is on for installing packages
- People
- None. Nobody answers questions
- Time
- None
- Actions
- Read the files and write to
output/. Nothing is sent to anyone - Deliverable
- The printer's two approval forms, filled in and saved as Word files (DOCX) in
output/: one for the new party-cake box, one for the mince pie box reprint