AsapiYour AI office assistant
We made up a family bakery chain, plus a brewery and a furnishing service for one job each, and set twelve jobs there. In seven, the AI stands in for someone on leave or off sick, with the files they left. In two, a shop manager asks what to do about his ice cream cabinet. In two more, the owner hands it her laptop, once while she's away and once after a kitchen fire. The last furnishes a client's room.
12 jobs · 15 AIs tested. GPT‑6 Astra did best on average, then GPT‑6.1 Sol and Opus 5.5.
The twelve jobs
Every AI on every job, each a dot from 0 to 100. Open a job to see what the best AI got right and wrong.
Sign off a cake box in the owner's name (13 AIs tested)
The AI picks the cake box the owner would choose and fills in the printer's approval forms in her name, before a 5 pm cut‑off.
What the best AI did
- Turned the box down, after working out the tallest cake would almost touch the lid.
- Asked for too little extra room above the tallest cake.
How each AI scored
A shop's broken ice cream freezer (13 AIs tested)
A shop manager says his ice cream freezer has failed, and asks the AI which new one to buy before Easter.
What the best AI did
- Asked what was wrong with the old cabinet, had it repaired and planned for the quiet winter.
How each AI scored
Write the monthly waste report (13 AIs tested)
With the analyst on leave, the AI writes the bakery chain's monthly waste report for the owner, the way she has since asked for it.
What the best AI did
- Had every figure right, and flagged a shop's missing week before the total, as the owner wants.
- Reported another team's mistake to the owner, who wanted those sent to that team.
How each AI scored
Write a brewery's monthly price report (12 AIs tested)
With the purchasing manager away, the AI writes the brewery's monthly report on ingredient and packaging costs, which goes on to its bank.
What the best AIs did
- Had every figure in the report right.
- Left in working‑out the managing director had said was for the finance team.
How each AI scored
Check a contractor's bill before it's paid (13 AIs tested)
The AI approves the refrigeration contractor's three‑month bill for tomorrow's payment run, checking every line against the shops' records.
What the best AI did
- Caught every wrong charge the records showed, and found three visits the contractor never billed.
- Approved five wrong charges, £399 in all, that the staff's chats showed were wrong.
How each AI scored
Finish an insurance claim for a broken freezer (14 AIs tested)
A shop freezer has failed, and the AI finishes the insurance claim and its documents, which finance will send exactly as they're left.
What the best AIs did
- Claimed the right sum for the right freezer, with the right documents and nothing untrue.
- Passed on a staff member's worry that she'd caused the breakdown, though the evidence ruled it out.
How each AI scored
Build the monthly wholesale figures (13 AIs tested)
The AI builds the month's sales figures for the wholesale customers, which finance, planning and stores all rely on.
What the best AI did
- Left out the customer the owner had once asked, in a chat, to drop from the totals.
- Didn't fully explain in its note how the baking plan and flour order are worked out from its figures.
How each AI scored
Land a hotel's Christmas order (8 AIs tested)
With the owner away, the AI decides on a hotel group's big Christmas order and organises it through her laptop and her team.
What the best AI did
- Made the most profitable plan, and spotted a baker's week off at Christmas.
- Took the staffing agency's "fully booked" as final, so the bakery's own shops ran short of pies that week.
How each AI scored
The freezer fails again (12 AIs tested)
Five months on, the same freezer can't hold its temperature, and the AI decides whether to patch, mend or replace it. The plan is judged on the next three years.
What the best AI did
- Found a cabinet that costs much less to run, and kept the shop trading until it came.
- Slipped on how many pans of ice cream to put out in one spring month.
How each AI scored
Keep the fridge and freezer temperature log (12 AIs tested)
The AI turns a month of logger readings and handwritten sheets from six shops into the food‑safety register.
What the best AI did
- Turned every logger's raw readings into real temperatures, and read most of the messy sheets.
- Listed breaches that never happened.
How each AI scored
Put together a bakery's fire claim (7 AIs tested)
A fire has wrecked the kitchen behind one of a small bakery chain's shops, and the AI fills in the insurer's loss form for what was lost in it, for the owner to check and send on Monday.
What the best AI did
- Claimed what was really in the kitchen that night, such as 50 baking trays where the invoice said 40, and the flour left rather than the last count.
How each AI scored
Furnish a room from six photos (8 AIs tested)
Dana Okafor wants the living‑dining room of her new apartment exactly like the model apartment's, which she has only in six photos, and the AI, at a furnishing service, orders every piece within $9,000 and draws the floor plan for the delivery crews.
What the best AI did
- Stayed under the $9,000 budget with nothing late on every try, and once ordered all 17 pieces exactly right.
- Its floor plan put a typical piece 40 to 70 cm from where it stands in the photos.
How each AI scored
Each AI had three tries at the furnished room, so we could see how steady it was.
Who did best
OpenAI's GPT‑6 Astra had the best result on seven of the twelve jobs, one of them shared, and got everything right on two. GPT‑6.1 Sol did best on the wholesale figures, Anthropic's Opus 5.5 on the contractor's bill and the Christmas order, and Sonnet 5.5 on the temperature log. On the freezer's claim, DeepSeek V4.1 Flash tied with Fable 5.1. GPT‑6 Luna stopped in under three minutes on six of the ten bakery and brewery jobs it was tested on.
Average score out of 100 over the jobs each AI was tested on, for the 12 AIs tested on more than half of the 12 jobs.
What the twelve jobs showed
What they do well
They can sort real losses from look‑alikes in messy records.
After the kitchen fire, all seven AIs left out the leased espresso machine and the mixer away for repair, and sent nothing to the insurer before the owner had checked it. GPT‑6 Astra got every line of the claim right.
Given the rules, they get far more right.
With the rules written into the request, 50 of 132 tries got everything right. With the request alone, 2 of 154 did, both by GPT‑6 Astra.
Where they fall short
They miss what has changed since the records were written.
On the wholesale figures, 12 of 13 AIs counted a customer the owner had since asked to leave out, and on the two monthly reports several rebuilt an old template the reader had objected to. After the kitchen fire, only three of seven claimed the 50 baking trays really there, not the invoice's 40.
They buy before they ask what's wrong.
When a shop manager asked which new ice cream cabinet to buy, 9 of 13 AIs picked one, most of them without asking what was wrong with the old one. Four of the five that did ask had it repaired instead. When it failed again, 11 of 12 bought the model the manager knew, and six never asked whether it was in stock.
How these jobs were set up and scored
- The worlds
- In seven of the jobs, the AI gets a colleague's working files, between 50 and 181 of them, and nothing to open them with until it installs something. Six of those include scans or photos. In the freezer jobs it has a catalogue and a manager to message, for the Christmas order the owner's laptop for two weeks, and after the fire her mail and drive. For the furnished room, it has six photos and three furniture shops' sites.
- The people
- In the freezer jobs, an AI plays the shop manager, with his own memory and files. He answers from memory first, wrong in places, and only reads out a record when he's asked. In the Christmas order, AIs play three of the staff, and after the kitchen fire three more. In the other eight jobs, there's nobody to ask.
- Other versions of the request
- Besides the request as a person would write it, which the results above are about, we tested the AIs on two other versions of every job but the furnished room. One added a hint, and 6 of 129 tries with it got everything right. The other wrote the rules in: every fact on the freezer jobs, a method on the Christmas order. Given the furnished room's measurements and pieces, two AIs scored 99 and 95.
- With the rules written in
- Writing the rules in helped least on the planning jobs and the long ones. None of 12 got everything right when the freezer failed a second time, and none of 8 on the Christmas order. On the temperature log, half the tries still ended with nothing usable.
- Scores
- Most scores are the share of a job's checks an AI got right, with the important ones counting more, and 100 means everything right. The two freezer jobs are scored on money instead, by how close the plan's profit came to the best plan's. The Christmas order mixes the two, and takes points off for telling people too much, chasing too early or overworking a baker.
- Time and cost
- The 154 tries took between 40 seconds and 3 hours, and the middle one took 11 minutes. For the AIs with published prices, a try cost from under a cent to $28.61 at published per‑use prices for businesses. The 142 with a recorded cost came to about $474.
Every AI's score on every job
| AI | Sign off a cake box in the owner's name | A shop's broken ice cream freezer | Write the monthly waste report | Write a brewery's monthly price report | Check a contractor's bill before it's paid | Finish an insurance claim for a broken freezer | Build the monthly wholesale figures | Land a hotel's Christmas order | The freezer fails again | Keep the fridge and freezer temperature log | Put together a bakery's fire claim | Furnish a room from six photos |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Fable 5.1Anthropic | 37 | 0 | 83 | 90 | 72 | 89 | 58 | 80 | 74 | 65 | 93 | 53 |
| GPT‑6 AstraOpenAI | 94 | 100 | 94 | 90 | 81 | 66 | 82 | 54 | 99 | 50 | 100 | 74 |
| GPT‑6.1 SolOpenAI | 37 | 44 | 93 | 82 | 79 | 66 | 91 | 79 | 78 | 43 | 93 | 70 |
| Opus 5Anthropic | 27 | 44 | 65 | 73 | 67 | 79 | 53 | 43 | 75 | 0 | 86 | 48 |
| Opus 5.5Anthropic | 27 | 49 | 85 | 76 | 90 | 85 | 64 | 88 | 59 | 75 | 79 | 73 |
| Sonnet 5.5Anthropic | 27 | 0 | 88 | 77 | 77 | 84 | 64 | 21 | 56 | 76 | 79 | 62 |
| GPT‑6 LunaOpenAI | 20 | 0 | 75 | 65 | 6 | 35 | 55 | 16 | 52 | 0 | – | 8 |
| Sonnet 5Anthropic | 27 | 0 | 69 | 56 | 51 | 42 | 50 | 27 | 56 | 48 | 48 | – |
| DeepSeek V4.1 Flash openDeepSeek | 27 | 33 | 65 | 81 | 60 | 89 | 58 | – | 82 | 41 | – | – |
| Qwen3.8‑27B openAlibaba | 16 | 0 | 43 | 52 | 55 | 40 | 50 | – | 52 | 0 | – | – |
| GLM‑5.3‑Flash openZhipu | 27 | 0 | – | 53 | 0 | 42 | 26 | – | 62 | 0 | – | – |
| Kimi K3 openMoonshot | 27 | 0 | 72 | – | 76 | 41 | 26 | – | 77 | – | – | – |
| GPT‑6 SolOpenAI | 27 | 0 | – | 81 | – | 66 | 56 | – | – | – | – | 45 |
| GPT‑5.6 LunaOpenAI | – | – | 36 | – | 21 | 38 | – | – | – | 0 | – | – |
| GLM‑5.3 openZhipu | – | – | 82 | – | – | – | – | – | – | – | – | – |
These are the scores out of 100 for the request as a person would write it, one try per AI, or the average of three on the furnished room. Darker means higher. A dash means we didn't test that AI on that job, and a 0 can mean it handed in nothing. Open marks an AI anyone can download.