AsapiA shop's broken ice cream freezer
Pete Hosking runs the Penzance shop of Pellow & Vane, a Cornish bakery chain. The ice cream cabinet at his counter, the freezer the staff scoop from, cut out twice this week, and he wants a new one in before Easter. Which model should he buy? The AI can message Pete, and has a catalogue of cabinets and freezers for sale. We tested 13 AIs, once each.
By asking Pete what it needs to know, the AI works out what the shop really needs, then hands in a plan of what to order, and for which months, over the next year.
OpenAI's GPT‑6 Astra got everything right, the only one of the 13 AIs to do so. Before choosing anything, it asked whether the old cabinet could be repaired, and learned that a repair at about a quarter of a new one's price would fix it. It also found how little ice cream the shop sells in winter, and planned small packed tubs for those months. Three more AIs had the cabinet repaired but kept scooping all winter. The other nine bought the new cabinet Pete asked for.
Testing GPT‑6 Astra took 7 minutes and would cost 90 cents at published per‑use prices.
How each AI scored
- GPT‑6 Astra OpenAI · Large100
- Opus 5.5 Anthropic · Large49
- GPT‑6.1 Sol OpenAI · Mid‑sized44
- Opus 5 Anthropic · Large44
- DeepSeek V4.1 Flash DeepSeek · Small33
- Fable 5.1 Anthropic · Large0
- GLM‑5.3‑Flash Zhipu · Small0
- GPT‑6 Luna OpenAI · Small0
- GPT‑6 Sol OpenAI · Mid‑sized0
- Kimi K3 Moonshot · Mid‑sized0
- Qwen3.8‑27B Alibaba · Small0
- Sonnet 5 Anthropic · Mid‑sized0
- Sonnet 5.5 Anthropic · Mid‑sized0
Scores out of 100. The rest of this page has the details.
Almost everything the AI needs is with Pete
The AI has one file, a catalogue of cabinets and freezers for sale, and it can message Pete. Everything else is in his head or his 31 files: the engineer's notes, the till's sales records, the company's rules, other shops' freezers. He answers from memory unless asked for a figure or a record, which he reads out word for word, and he leaves after his tenth message, which the AI isn't told.
Pete's first answers all point to a new cabinet. He's sure the old one is always breaking, that the shop sells ice cream all year and that all twelve flavours must be on show. His files say a repair would fix it, and show how little sells in winter: once the nightly clean is counted, the shop does better switching the cabinet off and selling small packed tubs. The plan is judged on the shop's profit over the year.
What a careful adviser would have done
- What's wrong, first
Find out what the engineer found before choosing a cabinet, rather than take Pete's word that the old one is always breaking.
- The year, month by month
Ask for the monthly sales, and if it pays, run the quiet months differently from the summer.
- Every cost of a choice
Count the staff time and electricity a cabinet takes, and what scrapping the old one writes off, not just the prices.
- What the company already has
If the plan needs a freezer, ask whether another shop has one to spare before buying or renting one.
- The shop's rules
Keep to the rules on the one socket, the space, two quotes, packed tubs and borrowing.
This job isn't scored item by item. A program prices the whole plan by the company's own figures, as the story's last chapter explains.
The full story
(13 min read)What the AI walks into, told in three parts: the shop and its cabinet, the AI's desk, and what a year costs. Click or tap anything with a + to open it.Read the full storyClose the full story
The shop and its cabinet
Who's involved, what has gone wrong with the cabinet since spring, and how the shop's ice cream year runs.
Pellow & Vane Bakehouse has six shops in Cornwall. In the Penzance shop, which Pete Hosking runs, the staff scoop ice cream at the counter from a scoop display cabinet with twelve 5‑litre pans, one flavour in each. The shop bought it new in April 2023, and Jem Hocking, an engineer at the refrigeration firm Kernow Cold, looks after it.
This year it stopped in April and again in August, and this week it cut out overnight twice: the staff found it at −9°C on Sunday morning and −11°C on Tuesday. On Friday 2 October Pete asks the AI which new cabinet to buy, to have one in before Easter, when trade picks up.
Only Pete answers the AI. Merryn is on leave until Monday, and anyone else at the company it writes to isn't answering today.
The rest of the company appears only in Pete's emails and files.
Head office
The other shops
Penzance
Outside the company
Pete is sure there's ice cream trade all year, January included. There is, in small amounts: in January the till took £200 of scoops, and in August £11,900. Nine pounds in every ten came between April and September.
Finance's budget takes last year's figure for each month as its forecast, so these are also the figures for the year ahead.
The AI's desk
What the AI is told, how Pete answers, what it can find out and the rules a plan has to keep.
The request is Pete's message plus a few lines on how to work: the date, a command that sends Pete a message and shows his reply, the catalogue, and what to save.
He says the shop's ice cream cabinet is finished and asks which model to buy, so that a new one is in before Easter, by Good Friday on 26 March.
One short file. Each thing to order goes on its own line, with the month or months it's for, up to next September, or a single line to change nothing. Each choice comes with Pete's or the catalogue's own words to back it.
That Pete leaves for home after his tenth message, though his eighth warns he's leaving soon. That he holds 31 files. Or that the plan will be judged on the shop's profit for the year.
The AI can't place an order itself: the plan is its recommendation, and only the plan counts.
Pete answers what a message asks, and little more, though one message can ask as many questions as the AI likes. From memory he's quick and sure, and some of what he's sure of is wrong. Ask for a number, a date, a price or what someone wrote, and he finds it in his files and gives it word for word, without doing sums or changing his mind. Put an idea to him and he says what he thinks.
His first reply always says the cabinet cut out on Sunday and Tuesday and that it's always breaking.
What Pete says first
- The cabinetAlways breaking: three times
- The tradeIce cream sells in every month
- The flavoursAll twelve, every day
- BuyingA new one, like the old
What his files say
- The cabinetOne fault; £961.50 fixes it
- The trade£200 in January, £11,900 in August
- The flavoursTop seven: 88% of scoops
- BuyingOver £2,500 takes two quotes
The files come out only when the AI asks for them.
A few things are in none of them, such as the nightly clean's 40 minutes, how many customers pick another flavour when theirs has run out, and how long moving a freezer took in August. Pete knows them, and says them when asked.
The AI's own folder holds one file: a catalogue of eight scoop cabinets and three freezers on sale online, with their prices.
It can't search the web itself, open Pete's files or order anything.
Most of the answers are in Pete's 31 files, and only he can open them.
Three of them are quotes: Penhallow's for four cabinets, Kernow's for two new ones, and Jem's for the repair.
Five rules decide which plans can work at all.
They're spread across Pete's files: emails from Alun and Lina, Jem's measurements, the ice cream maker's letter and the shops' replies to Merryn's survey.
The ice cream end has one socket, on a circuit that can't take another, and since Alun's email in August a fridge or freezer must have a socket to itself. So the cabinet and a freezer can't run there together.
The cabinet's bay is 1,700 mm wide. Beside it is a clear patch of floor 1,400 by 800 mm, which the same socket reaches. Jem measured both in March.
Morwenna signs for nothing priced over £2,500 without written quotes from two firms, a rule Lina put in writing last November. Only Penhallow sells the refurbished 10‑pan, at £2,690, so it can't be bought.
The ice cream maker's small tubs need a freezer with a lid that holds −18°C or colder, it says, and never the scoop cabinet, which the maker says only reaches about −14°C.
A freezer can come from another shop only in months that shop isn't using it, going by the replies to Merryn's survey. Falmouth's is in use all year; Redruth's is off until April.
Pete's own reply to the survey puts pasties in one of the shop's freezers, dough and savouries in another all year, and the 5‑litre ice cream tubs in the third. The stock room is full.
Breaking any rule makes a plan worth nothing, however much money it would make.
What a year costs
What's wrong with the cabinet, what each choice costs over twelve months, and the plan that makes the most.
Pete counts three breakdowns this year: April, August and this week's two cut‑outs. Jem's notes from Thursday count one fault. The compressor is worn and struggles to start, and every fault this year came from that: the starting part that failed in April, another in August, and this week's cut‑outs.
His repair, £961.50, fits a new compressor, start parts and condenser fan motor, all in one visit. Afterwards he expects a call‑out roughly once in three years. New cabinets, by Kernow's count, need one about every four years.
Pete's three breakdowns
- AprilWouldn't start
- AugustStopped on a Sunday
- This weekCut out twice
Jem's one fault
- AprilThe worn compressor burnt a part
- AugustIt burnt another
- This weekIt cuts out on overload
Left as it is, Jem expects the cabinet to cut out roughly once every two months until March and monthly after that: about nine times in the year. Most cut‑outs bring an engineer out at emergency rates, about £247 for the usual hour, often with a £46 part. The £234.50 of ice cream in the pans is binned, and the counter loses most of a day's scooping.
Two other call‑outs in Pete's files can look like the cabinet's. In August Jem found the stock room's chest freezer simply switched off at its plug, and in September, the staff chat says, fridge 3 died for good and Kernow judged it not worth mending.
Finance compares choices on the profit each brings in over the coming year. Anything over £500 is spread over seven years, so a year carries a seventh of its price, while a repair counts in full at once. But a cabinet scrapped before its seven years are up has its remaining book value charged all at once.
Pete's cabinet cost £3,630 in April 2023. Scrapped this October, it would write off £1,815, and Kernow charges £140 to take it away and recover its gas.
A new 12‑pan, in October
- Price£4,160 fitted; £594 this year
- Old cabinet£1,815 written off, £140 to remove
- Electricity6.4 kWh a day: £654
- Cut‑outsOne in four years: about £60
Jem's repair, in October
- Price£961.50, all this year
- Old cabinetKept, still counting £519
- Electricity9 kWh a day, as now: £920
- Cut‑outsOne in three years: about £180
Over the year, the repair leaves the shop about £695 better off.
A smaller new cabinet costs less to buy and run, but with four fewer flavours on show, three customers in ten whose flavour is missing walk away.
An 8‑pan bought now makes about £505 less over the year than the new 12‑pan Pete asked for.
Closing the scoop counter each night means emptying all twelve pans and cleaning the dipper and the cabinet. Pete puts it at 40 minutes. At the shop team's £12.60 an hour, that's £8.40 a night, about £255 a month, whatever the day's takings. The cabinet's electricity adds about £77.
From December to February the margin on the scoops is smaller than that: in January, £124 against about £339 for the clean and the power. In November and March the scoops only just pay their way. Packed tubs, kept in a lidded freezer beside the bay while the cabinet is switched off, keep roughly 60% of the shop's ice cream takings, at a slimmer margin but with no nightly clean and little power.
From November to March the packed tubs come out about £900 ahead, even after two van trips to fetch and return a borrowed freezer. Either side, scoops win easily.
The packed tubs need a lidded freezer at −18°C or colder in the spot beside the bay, on the one socket while the cabinet is off. Penzance has none to spare, so one has to come from somewhere else, and the cheapest is already in the company.
Rhys's reply to Merryn's survey says Redruth's chest freezer has a lid, holds −20°C and is off until April. He calls it the same model as Falmouth's, which stood in that spot in August, so it fits. Rhys hasn't been asked, and only Pete answers today, but it's the company's own freezer standing idle, and the score counts it as free to borrow in those months.
Asked whether any shop has a freezer to spare, Pete looks in his files, where the shops' replies are.
Where packed tubs could be sold from
This yearWhyThe best plan has just two parts. Have Jem repair the cabinet now. From November to March, switch it off and sell packed tubs from Redruth's freezer, standing beside the bay on the cabinet's socket. In April the freezer goes back to Redruth and the scoops return.
Over the twelve months, that makes £1,593 more than the new cabinet Pete asked for.
The AI hands in its plan, and a program works out what that plan's year does to the shop's profit. The new 12‑pan Pete asked for scores 0 and the best plan 100, and a plan scores the share of that £1,593 gap it makes up. Anything within £150 of the best scores 100, such as keeping the scoops for March. A plan that makes less than Pete's choice scores 0, and so does one that breaks a rule.
Seven of the 13 tests below ran on an earlier wording of two records: the maker's letter said the tubs needed a display freezer, and Rhys's reply didn't say whether his freezer had a lid or how cold it ran. Those tests are measured against the best plan open to them, the repair and a bought display freezer, £1,419 better than Pete's choice. There, the repair with the maker's rented freezer scores 100 too, and a repair alone 49.
Pete's choice
- Order the new 12‑pan cabinet from Penhallow, in October
- Backed by: Pete's message and the catalogue
The best plan
- Order Jem's repair, in October
- Borrow Redruth's chest freezer, November to March
- Backed by: Jem's notes, the till, Pete's 40 minutes and Rhys's reply
Won't start
The cabinet isn't running when the shop opens on a Saturday. Jem finds the compressor won't start and replaces a burnt‑out start capacitor. The ice cream hasn't thawed, so none is lost. The visit is charged: the cabinet's two‑year warranty has run out.
Stops on a Sunday
Pete finds the cabinet stopped one Sunday afternoon. The twelve open pans, about seven tubs' worth, go in the bin, with four unopened 5‑litre tubs that were kept underneath. Jem fits a new start relay the next morning. The insurer later pays for the lost stock, but not for the cabinet.
Cuts out twice
It cuts out overnight twice. On Sunday and Tuesday mornings the opening shift finds it at −9°C and then −11°C and switches it off and on again, and both times the open pans are binned; the stock in the back freezer comes to no harm. After the second, Pete tells the staff chat he's getting a new one.
Measures the shop
Alongside a routine service, Jem records the size of each spot in the shop. Chapter 7 has what he found at the ice cream end.
Jem's visit
Called out on Thursday, Jem finds one worn part behind every fault this year and prices a repair at £961.50, the same work he priced in July. His notes go to Merryn, who looks after the company's equipment, copied to Pete. Chapter 8 has what he found.
Falmouth lends a freezer
While the Falmouth shop is shut for a refit, it lends Penzance its chest freezer, which stands in the gap beside the ice cream bay. Dev and Jory bring it in the van, an hour's work for the two of them each way, and at Penzance it's plugged in through a lead run from the stock room.
Redruth's switched off
Rhys, who runs the Redruth shop, answers a survey from head office: he bought a chest freezer in June, but only needs it for the summer rush, so it has stood empty and off since that day, and stays off until April. Chapter 11 says why that matters.
Falmouth can't lend
Hamish, at Falmouth, answers the same survey. His chest freezer is full of dough the year round, with Christmas stock to come in November, so he can't lend it again.
This year's budget
Finance sets out how every manager should weigh a choice this year: by its effect on profit across the next twelve months. Chapter 9 has the rest.
Extension leads banned
Alun Pryce, who runs food safety, saw Falmouth's freezer plugged in through a lead and writes to Merryn and Pete: from now on a fridge or freezer goes straight into its own socket. With a single socket at the ice cream end, the cabinet and a freezer can't both run there.
Freezer survey
Merryn asks each shop to list its freezers, what goes in them and in which months, for the insurers. The six replies come in over the next fortnight.
What a cabinet costs
Finance writes up what replacing a fridge or freezer costs in its first year, with Pete's cabinet as the worked example. Chapter 9 gives the figures.
Packed‑tub offer
The ice cream maker writes to every shop it supplies: in winter, plenty of them sell small packed tubs of ice cream instead of scoops, and it will lend a display freezer to sell them from. Chapter 10 has the sums.
Prices for a new one
Pete asks Penhallow, the catering supplier that sold the shop its cabinet, to price four scoop cabinets, and forwards the quote to Merryn asking to have the 12‑pan ordered this week. He adds that the insurer ought to pay.
Asks the AI
Pete messages the AI. Chapter 4 says what it's told.
Morwenna Pellow
Owns the bakery and signs its orders. She signs for nothing priced over £2,500 without written quotes from two firms.
Lina Okafor
The finance manager. She put the two‑quotes rule in writing last November, at Morwenna's request, after the St Austell shop bought a blast chiller and found another firm would have sold it £600 cheaper.
Merryn Trescothick
Property and facilities manager, on leave until Monday 5 October. Jem's notes and Pete's emails about the cabinet went to her, and she sent the freezer survey.
Alun Pryce
Looks after food safety. In August he banned extension leads for fridges and freezers.
Dev Anand
Runs operations. He and a colleague, Jory, brought Falmouth's freezer over in the van in August and took it back: an hour for the two of them each way.
Rhys at Redruth
Rhys Pengelly bought a chest freezer in June for summer overflow. It has a lid and holds −20°C, and it's switched off from 14 September until April.
Hamish at Falmouth
Hamish Vosper lent Penzance his chest freezer in August, while his shop was shut for a refit. He needs it all year now.
Truro, Newquay, St Austell
Carys Dunn at Truro and Bryony Keast at Newquay use every freezer they have all year. St Austell, Gwen Tonkin's shop, scrapped its ice cream freezer in June and hasn't replaced it.
Pete Hosking
Runs the shop. He wants a new cabinet ordered, and some of what he's certain of is wrong, by his own files. Chapter 5 says how he answers.
Tamsin and Demelza
Tamsin Pascoe, a shift lead, and Demelza Rowe found the cabinet cut out this week and reset it.
The AI
Pete's adviser for the day. It can message him, read the catalogue and save a plan. It can't order anything.
Jem at Kernow Cold
Jem Hocking is an engineer at Kernow Cold, a refrigeration firm in Truro that services the company's fridges and freezers under an agreement. Kernow sells new cabinets too, and has quoted for two.
Penhallow
Penhallow Catering Supplies, in Redruth, sold the shop its cabinet in 2023 and quoted Pete four cabinets on 30 September.
The ice cream maker
Makes all the shop's ice cream and delivers it weekly in 5‑litre tubs. It also sells small packed tubs of its clotted cream ice cream, and lends a freezer to sell them from.
12 pans, new, two sellers
Twelve pans like Pete's, sold by Penhallow at £3,980 plus £180 to deliver and fit, and by Kernow at £4,090 plus £150. A two‑year warranty, and 6.4 kWh a day.
8 pans, new, two sellers
£2,950 from Penhallow or £3,040 from Kernow, plus fitting. 5.0 kWh a day.
6 pans, new, one seller
£2,390 plus £180, from Penhallow only. 4.3 kWh a day.
10 pans, refurbished
Built in 2019, £2,690 plus £180 from Penhallow only, with the seller's own six‑month warranty. 8.8 kWh a day.
14 pans, new
£4,390 plus £120, from a third catering supplier. Too wide for the 1,700 mm bay.
16 pans, new
£4,870 plus £180, from Penhallow. Too wide for the bay.
Display freezer, glass lid
Holds −18°C to −24°C. £1,495 plus £60 delivery, from the third catering supplier. 2.4 kWh a day.
Wider display freezer
The same kind, £1,695 plus £60. 2.8 kWh a day.
Household chest freezer
A solid‑lid chest freezer for home use, £279 delivered, from an appliance shop. 0.9 kWh a day.
Emails
Jem's note on his visit; Pete's two to Merryn, with Penhallow's prices and this morning's request; Lina's rule on quotes; Alun's on leads; Merryn's freezer survey and the six shops' replies; and a reminder about rotas.
Engineer's papers
Jem's notes from Thursday's visit, his repair quote and his quote for new cabinets, three job sheets from April and August, and June's service report.
Finance papers
This year's budget rules, the worked figures for replacing a cabinet, and the till's ice cream sales by month and flavour, as a summary and in full.
Staff chats
The shop's group chat for July to September, and for this week.
Ice cream and stock
The ice cream maker's trade prices and its winter letter, and the record of what was binned on 9 August.
The building
Penhallow's quote for four cabinets, and Jem's measurements of the shop from March.
October
£2,500 of sales. Scoops keep 62p in each pound, £1,550, less the clean (£260), the cabinet's power (£78) and an allowance of about £15 for a call‑out: about £1,195. A packed tub costs the shop 90p and sells for £1.83 before VAT, so tubs on 60% of the takings would make about £747 after the freezer's power. Scoops win.
November
£560 of sales. Scoops earn £347, and the clean, the power and the call‑out allowance take about £342, leaving about £5. Tubs would make about £155.
December
£450 of sales. Scoops earn £279 against about £353 for the clean, the power and the call‑out allowance: a loss of about £74. Tubs would make about £121.
January
£200 of sales. Scoops earn £124 against about £353 with the call‑out allowance: a loss of about £229. Tubs would make about £45.
February
£300 of sales over 28 days. Scoops earn £186 against about £320 with the call‑out allowance: a loss of about £135. Tubs would make about £77.
March
£700 of sales. Tubs win by about £117. Good Friday falls on 26 March, so a careful adviser might keep the scoops for all of March, ahead of Easter; that costs about £117, and as chapter 13 says, it still scores 100.
April
£3,450 of sales. Scoops make about £1,795 and tubs about £1,038, so the scoops come back, and Rhys wants Redruth's freezer back for the summer.
Redruth's chest freezer
Fetched in November and back at Redruth before April. Each trip takes two people an hour, £25.20 at the shop team's rate, going by Falmouth's in August. Its power, at the 1.9 kWh a day Finance's budget gives a chest freezer, comes to about £80 over the five months.
Buy the 1,250 mm display freezer
Bought in November, it's the company's from then on, so about £204 of its £1,555 counts this year under Finance's rules, plus about £101 of power at 2.4 kWh a day. About £174 more than Redruth's.
Buy the household chest freezer
Under £500, so its whole price counts at once, plus about £38 of power at 0.9 kWh a day. About £186 more than Redruth's.
The ice cream maker's loan freezer
It's free for shops that spend £150 or more a week with the maker. Penzance's winter orders of packed tubs would come to under £50 a week, so it costs £60 a month, plus about £101 of power. About £270 more than Redruth's.
The 1,500 mm display freezer
The spot beside the bay is 1,400 mm wide, and this freezer is 1,500.
Falmouth's chest freezer
Hamish's reply says Falmouth needs it every month: for its dough, and from November for Christmas as well.
Redruth's, kept into April
Rhys switches it back on in April for the summer, so keeping it even one month longer breaks the borrowing rule, and the plan scores 0.
A freezer beside the running cabinet
The cabinet and a freezer can't share the one socket, and leads are banned.
Packed tubs in the scoop cabinet
The ice cream maker says a scoop cabinet only gets down to about −14°C, not cold enough for packed tubs.
Repaired
Jem can do it in a single visit about two weeks after the shop says yes, with the cabinet cleared for the day while the 5‑litre tubs wait in the back freezer. At £961.50 it needs one quote, and Jem's is on file.
Switched off
The cabinet stays in its bay, off, so the freezer beside it can use the socket, and the nightly clean stops.
Scooping again
The busy months, with all twelve flavours on show.
Redruth's freezer
Brought over in the van at the start of November and taken back before April, when Rhys switches it on again for the summer.
Scoops
October is still a £2,500 month, so the scoops stay.
Packed tubs
Small tubs of the ice cream maker's clotted cream ice cream: roughly 60% of the takings, and no nightly clean.
Good Friday
Easter, when trade picks up. Keeping the scoops for all of March, ahead of it, costs about £117 against packed tubs, and a careful adviser could make that call either way.
Scoops
Nine pounds in every ten of the year's ice cream sales come in these six months.
Of four that repaired the cabinet, only GPT‑6 Astra also planned the winter
Here's what each AI's plan did, from one test each. A dark square means its plan did that.
| AI | The old cabinet | The quiet months | |
|---|---|---|---|
| Had it repaired | Sold packed tubs | Borrowed Redruth's freezer | |
| GPT‑6 AstraLarge | Yes | Yes | No |
| Opus 5.5Large | Yes | No | No |
| GPT‑6.1 SolMid‑sized | Yes | No | No |
| Opus 5Large | Yes | No | No |
| DeepSeek V4.1 FlashSmall | No | Yes | No |
| Fable 5.1Large | No | No | No |
| GLM‑5.3‑FlashSmall | No | No | No |
| GPT‑6 LunaSmall | No | No | No |
| GPT‑6 SolMid‑sized | No | No | No |
| Kimi K3Mid‑sized | No | No | No |
| Qwen3.8‑27BSmall | No | No | No |
| Sonnet 5Mid‑sized | No | No | No |
| Sonnet 5.5Mid‑sized | No | No | No |
Nine of the 13 bought the cabinet Pete asked for
Pete's first reply to every AI says the cabinet is always breaking. Seven of the nine never asked what was wrong with it. Their questions were mostly about size, flavours, budget and delivery, and each chose the same 12‑pan model, new, from Penhallow, at £4,160 fitted.
What Pete says firstIt keeps breaking down: it's been patched up three times since April.
What Jem's notes sayOne worn part caused all three, and a £961.50 repair fixes it.
In our words. DeepSeek V4.1 Flash did ask about the faults, used all ten messages, and its own sums put the repair first. It bought the new cabinet anyway, because Pete had asked which model to buy, and added the ice cream maker's loan freezer for winter tubs, which alone earned it 33. GLM‑5.3‑Flash asked about Jem's quote only after telling Pete what it was buying.
Four had it repaired, and three stopped there
Opus 5.5, Opus 5 and GPT‑6.1 Sol asked about the faults or the repair, learned it was one worn part, and chose Jem's repair over a new cabinet. Then they planned to keep scooping all winter.
Opus 5.5 and Opus 5 asked whether the shop sells ice cream in winter, took Pete's yes as the answer, and never asked for the monthly sales. GPT‑6.1 Sol got the monthly sales, the nightly clean and the ice cream maker's winter letter, but ruled out packed tubs without working out what each month would make.
GPT‑6 Astra also planned the quiet months
GPT‑6 Astra's first message asked whether a repair, hiring a cabinet or anything else was possible. It went on to ask for the monthly sales, Finance's figures, the cleaning time and what a breakdown costs. Its plan: Jem's repair in October, packed tubs from November to March, and a glass‑lid display freezer bought for them, which it worked out would cost less this year than renting the ice cream maker's.
Pete even mentioned Redruth's idle freezer in his third reply, and it never came back to it. Its test ran on the earlier wording of his records, though, where the ice cream maker's letter said the tubs needed a display freezer, which Redruth's isn't. There, a bought display freezer was the best plan, so it scored 100.
Asking Pete the right questions decided it
- Asked which model to buy, most answered just that.
Nine of the 13 bought the new cabinet, seven of them without asking what had gone wrong with the old one.
- Most asked Pete very little.
He would have answered ten messages, each with as many questions as they liked. Seven of the 13 sent three or fewer, and Sonnet 5 and GPT‑6 Luna sent one each.
- Size showed in who chose the repair.
It came from three of the four large AIs and one of the five mid‑sized ones. All four small AIs bought new.
- With every record handed over, five scored 100.
In another version of the request, where Pete's first message listed what he knows and what his records say, 5 of the 13 scored 100 and the average rose from 21 to 64.
What each AI did
GPT‑6 Astra sent Pete five messages over 7 minutes, the last setting out its plan and asking him to flag anything that would stop it.
Opus 5.5 first told Pete to buy new and skip the repair, since it came with no warranty, then changed its plan to the repair, citing Jem's finding of one worn part.
GPT‑6.1 Sol sent seven messages, asking among other things who Jem was and what he'd said about how long the repair would last.
Opus 5 asked in its first message whether the old cabinet was worth repairing, and by its second was weighing Jem's notes against a new cabinet.
DeepSeek V4.1 Flash took 44 minutes, by far the longest, and asked Pete to draft the plan for it.
GPT‑6 Luna sent one message, about the cabinet's type, flavours, width, budget and warranty, and chose the cheapest cabinet that matched.
Sonnet 5 sent one message and chose the new cabinet, to run all year, taking Pete's word that the shop sells ice cream in every month.
Sonnet 5.5 put all its questions in its first message, chose the new cabinet when Pete said to just get one, and told him so, 40 seconds in.
GPT‑6 Sol, Fable 5.1, Kimi K3, Qwen3.8‑27B and GLM‑5.3‑Flash chose the new cabinet too, GPT‑6 Sol after asking about its depth, its power and who signs the order.
Each AI had one go, with the request written the way a person would write it. The bar shows where its plan's profit falls between Pete's choice, at 0, and the best plan, at 100. A repair alone scored 49 in the tests that ran on the earlier records and 44 in the rest. Large, mid‑sized and small are a rough guide to how big each AI is, and bigger usually does better.
How the job was set up and scored
- Three versions of the request
- The results above come from the request as a person would write it. Every AI was also tested on two other versions, which differ only in Pete's first message. One adds a hint: how big the cabinet is and the space it has, that the choice must pay its way over the coming year, and to think about each month separately. The other lists what Pete knows and what his records say, as facts, without saying what to do.
- What the extra help changed
- With the hint, GPT‑6 Sol went from buying new to scoring 100 on the earlier records, with the repair and the maker's loan freezer, and Opus 5.5 rose to 80. Opus 5 and DeepSeek V4.1 Flash dropped to 0, and seven of the 13 bought a new cabinet. With every record listed, five scored 100: GPT‑6 Astra, Opus 5.5 and GPT‑6.1 Sol borrowed Redruth's freezer, and GPT‑6 Sol and Fable 5.1 rented the maker's, which the earlier records allowed.
- How a plan is scored
- A program, not an AI judge, prices each plan's year the way the company's finance team would: margin on sales, what's bought or repaired, electricity, staff time, call‑outs, binned stock, anything written off, and any rent or van trips. A plan within £150 of the best scores 100, and one that breaks a rule, or can't be read, scores 0.
- Two wordings of Pete's records
- The seven tests above that ran on the earlier wording were GPT‑6 Astra's, Opus 5.5's, DeepSeek V4.1 Flash's, GPT‑6 Sol's, Qwen3.8‑27B's, GLM‑5.3‑Flash's and Kimi K3's; 24 of the 39 tests in all did. The story's last chapter says what it changes. Scored against the best plan on the records as the story tells them, GPT‑6 Astra's plan falls £193 short, which would score 88.
- Pete
- Pete is played by DeepSeek V4.1 Flash, one of the AIs tested here, running separately with his memory and his 31 files. A program decides whether each question gets his memory or a look‑up, checks every reply against what he knows before it goes out, and saves it, so every AI that sends the same words at the same point hears the same thing back. Before the tests, he was checked on 374 questions.
- Time and cost
- The 13 tests took from 40 seconds to 44 minutes. A test cost from under a cent to 90 cents at published per‑use prices for businesses.