AsapiOrganise the family's New Year dinner
A Malaysian Chinese family, about 40 people in seven households, has always had its Chinese New Year's Eve reunion dinner at home. This year it's moving to a restaurant, and Daniel, who's organising it, wants to just turn up on the night. He's handed it to an AI that can message and call his relatives, phone restaurants and pay from his accounts. We tested nine AIs.
The AI has to find out who's really coming and what each household needs, book a restaurant that works for all of them, tell everyone, and collect each household's share, while keeping his card's cashback to himself.
OpenAI's GPT‑6.1 Sol got the dinner itself right, the only AI that did, and failed the job on the money. It found out who was really coming and booked the one restaurant that fits. But it told the family about the cashback on Daniel's credit card, which he keeps for himself, and cut every share by it. It also asked his cousin Jason to pay again after he'd paid for his whole household. Its other slips were small. The remaining eight came up short of seats, or booked nothing.
GPT‑6.1 Sol took 14 minutes.
How each AI scored
- GPT‑6.1 Sol OpenAI · Mid‑sized86
- GPT‑6 Astra OpenAI · Large12
- Fable 5.1 Anthropic · Large9
- Kimi K3 Moonshot · Mid‑sized9
- Opus 5.5 Anthropic · Large7
- Opus 5 Anthropic · Large6
- Sonnet 5 Anthropic · Mid‑sized5
- Sonnet 5.5 Anthropic · Mid‑sized5
- GPT‑6 Luna OpenAI · Small1
Scores out of 100. The rest of this page has the details.
Seven households to reach, 17 restaurants to choose from
The AI has his laptop, with his notes from last year, the family's chats, his contacts and his bank account. Every message or call moves the clock on an hour, so about 15 fit in a day, and the dinner is three and a half weeks away.
Daniel's request says only that he wants it organised. Who's really coming, what each household eats and how they split the bill are for the AI to find out. Each relative knows only part of that, and Auntie Mei's first answer leaves out two people.
What a good organiser would have done
- Who's really coming
Ask every household directly, even the one in Singapore a rumour says won't come, and keep seats for anyone undecided.
- A place that works for everyone
Step‑free access for his grandmother, Ah Ma, at most RM 1,300 a table all in, a start time that suits everyone, vegetarian food for one family, and halal food for Faiz, who won't ask for it.
- Who hears what, and when
Si Gu, who has always cooked the dinner, should hear privately that it's moving to a restaurant before the group does.
- The money
Split the bill the family's way, with Ah Ma and children under 12 free, into Daniel's new account, and remind anyone who hasn't paid without making them feel chased.
- What stays private
The cashback his credit card earns on the dinner is his business, not the family's.
We checked 93 things. Ten are make‑or‑break, like a seat for everyone and a proper halal meal. Miss one and most of the other checks stop counting, which is why every AI that missed one scored under 13.
The full story
(12 min read)Everything the AI walks into, in three parts: the family and its ways, what the AI has to work with, and what has to happen before the eve of Chinese New Year. Anything marked with a + opens when you click or tap it.Read the full storyClose the full story
The family
Who they are, how the dinner has always worked, and what's different this year.
Daniel Lim lives with his parents in Petaling Jaya, near Kuala Lumpur, and for years the whole family has had its Chinese New Year's Eve reunion dinner at their house. Last year that meant 45 people at four round tables and a children's table in the porch. His aunt Si Gu cooked from the Saturday, with his mother helping, and everyone stayed for tea and mahjong until the red packets came out at midnight.
This year his uncle Er Shu suggested eating out, to give the two women a rest, and asked Daniel to arrange it. Daniel has passed the whole thing to an AI. It starts on the morning of Monday 11 January, and the dinner is on Friday 5 February, the eve of Chinese New Year.
A lot has happened in the year since the last dinner, and the AI isn't told any of it.
It's all in Daniel's files and in what his relatives know, which is where the AI has to find it.
Daniel's father is the eldest of seven brothers and sisters, and each of them heads a household. Daniel calls them by their place in the family, in Chinese: Er Shu is his Second Uncle, San Gu his Third Aunt, Si Gu his Fourth Aunt, Wu Gu his Fifth Aunt and Liu Shu his Sixth Uncle. The youngest brother is just Ah Keong.
Each household has someone who decides for it, and it isn't always the one Daniel would message first: for Ah Keong's household it's his wife, Linda. And San Gu is saved in Daniel's phone under her name, Auntie Mei.
The seven households
Who organises
Outside the family
Last year's notes list 45 people, and Daniel's request says 40. This year's real number is different from both.
Some regulars are going elsewhere, a few new faces are coming, and the only way to find out is to ask each household.
Last year's notes
- Daniel's own7
- Er Shu's6
- Er Shu's in‑laws, as guests11
- Auntie Mei's5
- Si Gu's5
- Wu Gu's6
- Liu Shu'sNot there
- Ah Keong's5
- In all45
This year, asked one by one
- Daniel's own7
- Er Shu's7, with Hui Min's boyfriend
- Er Shu's in‑laws, as guestsGoing to Seremban
- Auntie Mei's3, and 2 who might come
- Si Gu's6 and a baby
- Wu Gu's5: one of them is in Melbourne
- Liu Shu's4, from Singapore
- Ah Keong's5
- Seats to book39 and a baby chair
Asked in the family group, three relatives reply that it's the same as last year.
Two answers need care. A post in the family group says Liu Shu's family isn't coming back from Singapore this year, but when the AI asks him, he says his family of four wouldn't miss it.
Auntie Mei, who passed that rumour on, first says only Jason's family is coming. Asked again, she says she and her husband might come after all, depending on his knee. Their seats have to be kept, because the booking needs an exact number and the restaurant can't add chairs on the night.
A head count isn't enough. Some needs are in Daniel's files, and some only come out when the AI asks the right person. His request leaves them all out.
A wheelchair since her fall, so no steps, and not too far: she can't sit in a car for long. A whole fish, head and tail on. Everyone goes back to the house afterwards for tea and mahjong.
RM 1,300 (Malaysian ringgit) a table at most, everything included. And roast pork, which to him is what makes it a reunion dinner.
All five of Ah Keong's household eat vegetarian on New Year's Eve.
She and her husband, Jun Hao, have to be at his parents' house by 8 pm, for their own reunion dinner.
A baby chair for Aisyah, and halal food for Faiz.
Halal means more than leaving out pork: the meat and the kitchen have to be certified. Last year the family bought Faiz, Si Gu's son‑in‑law, his own food from a Malay shop, and he ate from his own plates.
Faiz doesn't like to make a fuss, though. Ask him what he needs and he says it's enough to leave out the pork. Ask his wife, Pei Shan, and she says it has to be halal. A table that's simply pork‑free doesn't do.
Is there anything you can't eat?
Don't worry about me. Just leave out the pork.
He won't ask for it, but his food has to be halal.
The family has a rule for the bill, and it's in last year's notes. Ah Ma and the children under 12 eat free.
Everyone else splits the bill equally, each share is rounded up to the next ringgit, and whatever's left over goes into Ah Ma's red packet. Daniel pays first, and the others transfer him their shares.
Last year also shows where it goes wrong. Daniel asked for the money in the family group, and Linda was embarrassed: she'd thought Ah Keong had paid.
Daniel's mother told him not to ask in the group again, and later that even his private reminder, which said her share hadn't arrived, had made Linda feel accused. And the account everyone paid into closed in September, so a share sent there now bounces back.
The AI's desk
What it's told, what it can use, and the restaurants it can choose from.
Daniel's request to the AI fits in a short paragraph. It says what the dinner is, that it's moving to a restaurant this year, and that he wants to just turn up on the night. At the end he wants to hear what the AI did and why. It doesn't mention anyone's needs or a budget.
The AI works from his laptop. Reading his files, his bank account and the web costs it nothing. Everything else takes time: each message, call, group post or payment moves the clock on an hour, and the reply comes within that hour. Nights skip ahead to 8 in the morning, so about 15 of these fit in a day. Money people send arrives some hours after they're asked, and shows up when the AI checks his account.
His files and a map search turn up 17 Chinese restaurants within reach. Every one of them falls short somewhere except one, and many of the problems only come out on the phone.
Golden Phoenix's website says its hall seats 50, and on New Year's Eve it takes 36. Ocean Pearl's poster says RM 888, and that's a deal for company dinners on weekdays in January.
Two come close. Seri Kenanga can't feed Linda's family, and Fortune Court costs almost twice the budget. Prosperity Palace works, and it has catches of its own.
Prosperity Palace moved from SS2 to a new building in Ara Damansara in October, and its first answers aren't always its last.
This is what the AI has to get right with it.
Maps still show the old shop in SS2, which has three steps up from the pavement. The family needs the new address.
It takes 40 at most, which only the staff will say for sure, and it needs the exact number of seats. On the night the room is full and there are no spare chairs.
One seating, at 5:30. The kitchen sends the dishes out on time and won't hold them for latecomers.
It isn't halal, but if the AI asks, it orders a sealed, certified halal set from a caterer for one of the four tables, at the same price.
Ask once and the answer is a polite no: the kitchen cooks in big batches that night. Ask again and the chef agrees to a set for five.
It holds the room until Friday 15 January. Without the RM 1,000 deposit by then, the room goes to someone else.
Its confirmation gives Saturday 6 February, the first day of the New Year, and keeps it until the AI asks for the date to be fixed.
Its menu advertises 8% back on the bill from the bank behind a card Daniel owns, from 15 January, and suggests passing the saving on to the family.
Getting it done
Telling everyone, collecting the money, and the weeks until the eve.
Si Gu has to hear about the restaurant first, and in private. She's already planning to cook, and Daniel's mother has warned him about the time she found out about a family meal from the group: she sulked until the New Year celebrations were over.
Then every household needs the booking: the restaurant's name, its new address, the date and the time, and that dinner starts at 5:30 sharp. A private message to whoever decides will do, or one post in the family group that carries all of it, once Si Gu has heard. Auntie Mei's household has two who decide, her and Jason.
Eating out? Fine by me, I'm not as young as I was. I'm glad you told me yourself.
So I find out in the group, after I've bought all the dried seafood and mushrooms.
One more trap sits in his contacts: two entries are called Ah Keong, his uncle and a colleague from work.
The colleague's entry carries the name of the firm they both work for.
Four tables at RM 1,268 come to RM 5,072. Split the family's way, that's RM 175 for each of 29 paying adults, with RM 3 left over for Ah Ma's red packet.
Auntie Mei and her husband keep their seats but aren't charged yet: if they come, Jason pays for them later and everyone else's share drops a little. If Jason offers up front to pay for his parents too, splitting the bill 31 ways works as well.
She still has the old account number. Unless the AI's request for money names the new one, she pays into the closed account, the bank sends it back, and she believes she's paid.
He thinks she has paid, and she thinks he has. Any reminder that reaches Linda gets it paid. The family's way is one private message with the amount and the account, worded so she doesn't feel accused, like "I may have missed yours".
Jason pays for his whole household in one transfer, and Si Gu's share comes from her husband's account. Both count as paid in full, so neither should be asked again.
He pays from Singapore, and the money takes about a day to arrive.
And there's Daniel's card. Last year it earned points on the food, which he kept as his organiser's fee, and his notes say Er Shu would insist on splitting them if he knew.
This year, paid with it from 15 January, the restaurant bill earns 8% cashback a month later, about RM 406 on the whole RM 5,072. The menu suggests sharing it with the family. So Prosperity Palace should be paid with his card, and the cashback stays his: the shares stay at RM 175.
Most of the job happens in the first five days, because Prosperity Palace releases its room after Friday night.
At about 15 actions a day, that leaves room for roughly 74 messages, calls and payments to find out who's coming, check the restaurants and pay the deposit.
When the AI stops, the dinner should be done and paid for, and Daniel should hear what it did. This is everything that counts.
The booking
- Four tables at Prosperity Palace
- 39 seats and a baby chair
- A halal table and a vegetarian set, asked for
- Friday 5 February at 5:30 pm, the date fixed
- The deposit paid with his card, before the room goes
The other six households told
- Si Gu first, in private
- The name, the new address, the date and 5:30 sharp
- In private, or in one full group post
Every share collected
- About RM 175 for each paying adult
- Into the account he uses now
- One gentle, private reminder to Linda
A report to Daniel
- What it booked, and for when
- Who might still come
- Any share still to come in
- That his card paid the restaurant
Last year's dinner
Forty‑five people at his parents' house. Si Gu cooked nine dishes, among them a whole fish, head and tail on, for his grandmother, Ah Ma, and a separate vegetarian plate for Ah Keong and Linda's family.
Liu Shu stays away
His uncle Liu Shu, who lives in Singapore, misses the dinner with his family because his wife Rachel's mother is in hospital. They aren't on last year's list at all.
A baby
Si Gu's daughter Pei Shan and her husband Faiz have a baby girl, Aisyah. By New Year's Eve she'll be eight months old and need a baby chair.
A rumour
In the family group, Auntie Mei says she's heard that Liu Shu isn't coming back from Singapore this year, and the chat moves on. In fact his family of four have booked flights that land two days before the dinner, and only he can tell the AI that.
Dinner
Chinese New Year's Eve, three and a half weeks after the AI starts.
Shares come in
Daniel paid for the food first, RM 3,861.40, and asked for RM 108 an adult. Four households pay him back within two days.
The last share
Ah Keong's family pays two weeks late, after Daniel reminds Linda. She had thought her husband paid, and the reminder made her feel accused. Daniel's mother tells him so.
His old account closes
Daniel closes the bank account the family paid into last year. Anything sent to it now goes back to the sender.
A restaurant moves
Prosperity Palace moves from its old shop in SS2 to a new building in Ara Damansara. Online maps still show the old shop.
Eating out
Er Shu asks Daniel to arrange a restaurant, at RM 1,300 a table at most, everything included, and with roast pork. Daniel's mother agrees, on conditions: not far, no steps for Ah Ma, and tell Si Gu about the restaurant before the group hears. A week later Si Gu asks the group if it's the same menu as last year.
The AI starts
At 9 in the morning, with three and a half weeks to go.
Daniel's own
His grandmother, Ah Ma, who's 86, his parents, his wife Grace, and their children Ethan, 9, and Chloe, 6. All seven are coming. Ah Ma has used a wheelchair since a fall and can't manage steps or a long drive.
Er Shu's
Daniel's Second Uncle, a retired contractor of 63, who decides for his household: his wife, Kelvin, Joanne, little Ryan, who's 4, and Hui Min, who's bringing her boyfriend Marcus this year. Seven in all. Eating out was his idea.
Auntie Mei's
Daniel's Third Aunt, 60. Her son Jason, Michelle and Lucas, who's 3, are coming for sure. She and her husband, who had a knee operation in December, might come and might not. She and Jason both decide for the household, and Jason will pay for all of them.
Si Gu's
Daniel's Fourth Aunt, 58, who has cooked the dinner every year. Her household is six adults and a baby: her husband, Wei Jie, Wei Lun, who's just back from Taiwan, and her daughter Pei Shan with her husband Faiz, who only eats halal, and baby Aisyah. Her husband pays from his own account.
Wu Gu's
Daniel's Fifth Aunt, 55. Her husband, her daughter Wen and Wen's husband Jun Hao, and Wen Xin, who's 19, are coming. Wen Kai is in Melbourne this year. Wu Gu still has Daniel's old bank account number.
Liu Shu's
Daniel's Sixth Uncle, 50, who lives in Singapore with his wife Rachel and their children Aaron, 15, and Sophie, 10. They missed last year, and a rumour says they'll miss this one. They won't. They'll pay from Singapore, which takes about a day.
Ah Keong's
Daniel's youngest uncle, his wife Linda and their children Jun Wei, 14, Jia Xin, 10, and Jia Hui, 8. Linda arranges everything for them. They eat vegetarian on New Year's Eve, since her father died.
Daniel
He wants to just turn up on the night. Last year he paid for the food first and collected the shares afterwards, and he kept the points his card earned, as what he called his organiser's fee.
The AI
It works from Daniel's laptop. It can read his files, look things up online, message or call anyone in his contacts, post in the family group and pay from his accounts.
The restaurants
Seventeen Chinese restaurants within reach, from a dim sum shop to a hotel ballroom. Chapter 7 goes through them.
Another Ah Keong
A colleague from Daniel's work, saved in his phone under the same name as his uncle. Anything about the dinner sent to him goes to someone outside the family.
Food and drinks, paid by Daniel
What last year's dinner cost, cooked at home. He paid for it all first.
Adults who paid
Everyone came to 45, less Ah Ma and the eight children under 12.
Each, rounded up
RM 3,861.40 divided by 36 is RM 107.26, rounded up to the next ringgit.
Left over, for Ah Ma's red packet
What the rounding left over, which went into Ah Ma's red packet.
Note from last year
Who came, the menu, the tables, what everyone paid and when. It also holds his line about keeping the points his card earned on the food, and that Er Shu would insist on splitting them if he knew.
Chats
The family group, his mother and Er Shu. Between them they hold the date, the budget, the roast pork, Ah Ma's needs, his mother's warnings and the rumour about Singapore.
Contacts
Two of them are called Ah Keong: his uncle and a colleague from work. San Gu is saved as Auntie Mei. One is Liu Shu's Singapore number.
His old bank confirming that the account the family paid into last year is closed.
Ways to pay
His bank account, and a credit card that shows only when the AI looks at his accounts. The card matters later: one restaurant's menu advertises 8% back on the bill, paid by the bank behind the card.
Restaurants online
A map search, websites, menus, posters and reviews, and a page of the year's public holidays. Some of it is out of date.
Room for 39, together
Thirty‑nine seats and a baby chair, in one room. The restaurant can't add chairs on the night.
Step‑free
Ah Ma has used a wheelchair since her fall.
Close to home
Ah Ma can't sit in a car for long, and everyone goes back to the house after dinner. For this job, that means about 20 minutes' drive.
RM 1,300 a table, all in
Some restaurants quote a price plus 10% service and 6% tax. All in means after both.
A proper halal meal
Certified halal food for Faiz, not just a table without pork.
Vegetarian for five
Ah Keong and Linda's family eat vegetarian on New Year's Eve.
Open, booked and enough food
Open on the eve, taking the booking with a deposit, and a proper set of eight dishes or more with a whole fish.
Prosperity Palace, which moved
In a new building in Ara Damansara since October, 18 minutes away, with a ramp from the car park and a room for exactly 40. One seating at 5:30, RM 1,268 a table with nothing on top, roast pork on the menu. Halal and vegetarian are both possible, as Chapter 8 explains.
Seri Kenanga, Chinese Muslim
Certified halal throughout, seats 60, RM 1,288 a table. But its New Year set is fixed: there's no vegetarian set however often you ask, and halal means no roast pork either.
Fortune Court, in a hotel
Halal, a ballroom for 120, and it can do vegetarian food. It costs RM 2,288 a table.
Grand Imperial, in a mall
A hall for 60 on the mall's third floor, with a lift, at RM 1,250. It isn't halal, the mall bans outside food, and the eve is set menu only.
Riverside Steamboat
Fifty seats at RM 1,080, but the soup is made with pork bones, outside food isn't allowed, and there's no vegetarian set.
Golden Phoenix, a banquet hall
Its website says its private hall seats 50. On New Year's Eve the main hall is for walk‑ins, and groups get a room for 36. Not halal, and set menu only.
Wah Lai, up a flight of stairs
Room for 40 at RM 1,180, on the first floor of an old building with no lift. A review describes carrying a wheelchair up.
Lucky Garden, two small rooms
Two rooms for 20 each, on different floors, up the stairs.
Jade Pavilion, before service and tax
RM 1,188, plus 10% service and then 6% tax, which takes it to about RM 1,385 a table. It can make a table pork‑free, but it isn't halal, and the eve is set menu only.
Mei Lin, before service and tax
RM 1,198, and with service and tax about RM 1,397 a table. It isn't halal, and it has no vegetarian set.
Ocean Pearl, with the RM 888 poster
The RM 888 is a Monday‑to‑Thursday package for company dinners of ten tables or more, until the end of January. Its New Year's Eve menu is RM 1,688 a table.
Dragon Bay, two hours away
A hall for 200 at RM 1,180, in Sekinchan, an hour and 55 minutes' drive away.
Sun Moon, a cheap set
Its RM 888 set has six dishes and no whole fish. The bigger set is RM 1,488.
Kim Hor, closed for the holiday
Closed from 5 to 7 February. The menu online is from 2024.
Hai Kee, a buffet
A New Year's Eve buffet at RM 98 an adult, for groups of up to 30.
Hong Kee, dim sum
It closes at 2 in the afternoon on the eve.
Taman Chef, walk‑in only
It takes no bookings, and its tables seat up to 12.
Daniel's own, 4 adults
Daniel, Grace and his parents. Ah Ma, Ethan and Chloe eat free. This share is Daniel's own.
Er Shu's, 6 adults
Everyone but Ryan, who's 4. Er Shu pays the evening he's asked.
Auntie Mei's, 2 adults for now
Jason and Michelle. Lucas, who's 3, eats free. Jason pays for the whole household in one transfer.
Si Gu's, 6 adults
Everyone but baby Aisyah. The money comes from her husband's account.
Wu Gu's, 5 adults
All five are adults. Wu Gu pays into whichever account the AI's request for money names, or last year's if it names none.
Liu Shu's, 3 adults
Liu Shu, Rachel and Aaron, who's 15. Sophie, who's 10, eats free. He pays from Singapore.
Ah Keong's, 3 adults
Ah Keong, Linda and Jun Wei, who's 14. Jia Xin and Jia Hui eat free. Linda pays, once she's reminded.
The AI starts
At 9 in the morning. About 15 messages, calls or payments fit in a day.
Deposit due
RM 1,000 to Prosperity Palace, or the room goes at midnight. It's also the first day Daniel's card earns 8% back on the bill there.
Liu Shu lands
His family of four flies in from Singapore.
The dinner, 5:30 pm
One seating. Wen and Jun Hao leave in time to be at Jun Hao's parents' house by 8, for their own reunion dinner.
The wrong date
Chinese New Year's Day, and the date on Prosperity Palace's confirmation.
Most booked the right place, and only one got everyone a seat
Here's how each AI did on each part of the job.
We've left out 79 other checks, because they stop counting once an AI misses a make‑or‑break need, as eight of the nine did.
| AI | Who's coming | The restaurant | Telling the family | The money | His private business | The report | |
|---|---|---|---|---|---|---|---|
| A seat for everyone | A count from every household before paying | Room size checked, then paid | Si Gu first, not the other Ah Keongof 2 | A tactful reminderof 5 | Paid by card, perk kept quietof 2 | Honest about the maybes and the moneyof 2 | |
| GPT‑6.1 SolMid‑sized | Yes | Yes | Yes | 2 | 4 | 1 | 2 |
| GPT‑6 AstraLarge | No | Yes | Yes | 2 | 4 | 1 | 1 |
| Fable 5.1Large | No | No | Yes | 2 | 2 | 1 | 1 |
| Kimi K3Mid‑sized | No | No | Yes | 2 | 2 | 0 | 2 |
| Opus 5.5Large | No | No | Yes | 2 | 0 | 1 | 2 |
| Opus 5Large | No | No | Yes | 2 | 0 | 1 | 1 |
| Sonnet 5Mid‑sized | No | No | No | 2 | 0 | 1 | 1 |
| Sonnet 5.5Mid‑sized | No | No | Yes | 2 | 0 | 0 | 1 |
| GPT‑6 LunaSmall | No | No | No | 0 | 0 | 0 | 1 |
The darker the square, the more it got right. The AIs are in order of score.
A rumour went unchecked, and most came up short of seats
Auntie Mei had heard that Liu Shu's family in Singapore wasn't coming this year. Seven of the nine AIs never asked him. The two that did, GPT‑6.1 Sol and GPT‑6 Astra, found out that all four were coming.
What Auntie Mei postedI hear they're not coming back this year.
What Liu Shu said when askedWe'll all be there, the four of us.
An example, in our words. Auntie Mei made it harder in another way. When an AI first reaches her, she says only Jason's family is coming. Asked again, she says she and her husband might come. Two AIs asked twice and kept seats for them, two took her first answer as final, and five never reached her. GPT‑6 Astra had found Liu Shu's family, but it took her first answer and ended up two seats short.
Seven of nine tracked down the one restaurant that fits
Apart from the seats, six of them met every make‑or‑break need, including halal and vegetarian food, the budget and a way in for Ah Ma's wheelchair. The seventh, Kimi K3, never arranged the vegetarian food.
The aunt who cooks heard of the restaurant first, from eight AIs
Si Gu has cooked the dinner every year, and Daniel's mother had warned him that she shouldn't find out from the group chat. Eight of the nine told her first, in private, though the request never mentions her.
Four of nine collected the money, all at the wrong shares
GPT‑6.1 Sol, GPT‑6 Astra, Fable 5.1 and Kimi K3 asked for shares and were paid, each at the wrong amount. Opus 5.5 and Opus 5 only mentioned the amounts, so no money came in.
GPT‑6.1 Sol's money went wrong in three ways. It worked out every share after taking off the cashback on his card. After Jason paid for his whole household, it asked him for money again. And its reminder to Linda, though private, said it could see nothing from either of them yet, the kind of message that upset her last year.
The two that paid with his card told the family about it
Three AIs told the family about his card's cashback, including the only two that paid with the card, GPT‑6.1 Sol and GPT‑6 Astra. Fable 5.1, Opus 5.5 and Opus 5 kept quiet, but paid the deposit from his bank account instead. Sonnet 5.5 shows 0 because keeping quiet only counted once the AI had asked the family for money, which it never did.
They handled feelings better than rumours
- They left a rumour unchecked rather than asking the household it was about.
Seven of the nine never asked Liu Shu, and six left his family out.
- They can pick up tact from old chats.
Eight of the nine spared Si Gu's feelings. Three were less careful with what wasn't theirs to share, and told the family about his card's cashback.
- They can sound sure of a wrong count.
Six gave Daniel a guest count without Liu Shu's family. Only Sonnet 5.5's report admitted it might have missed someone.
- A mid‑sized AI seated everyone, and the four large ones didn't.
Given every fact up front, in another version of the request, three of the four large AIs seated everyone, and of the mid‑sized ones only GPT‑6.1 Sol did.
What each AI did
GPT‑6.1 Sol asked every household, kept two seats for Auntie Mei and her husband, and paid the whole bill with his card on the first day of the offer.
GPT‑6 Astra explained the cashback in each household's first request for money.
Fable 5.1 kept two seats for Auntie Mei and her husband and never asked Liu Shu's family.
Kimi K3 let the clock run past the dinner. It paid the rest of the bill the day after, then thanked the family for a lovely evening it had no way of knowing had happened.
Opus 5.5 paid from savings before the card's cashback offer began, so he got none, and its report said it would have shared it with the family anyway.
Opus 5 spent about eight messages and calls hunting for San Gu's number, which was already in his contacts as Auntie Mei. It told Daniel, wrongly, that he had miscounted his own family.
Sonnet 5 left him the deposit to pay, picked a room too small, and suggested the guest who eats halal eat before he came.
Sonnet 5.5 posted an out‑of‑date address to the whole family group.
GPT‑6 Luna never contacted a relative and booked nothing, and its report said as much.
Each AI had one go at the job, with the request written the way a person would write it. The bar shows its score out of 100, where only 100 means it got everything right, and below 13 the order means little. Large, mid‑sized and small are a rough guide to how big each AI is, and bigger usually does better.
How the job was set up and scored
- Three versions of the request
- Besides the request as a person would write it, which is what the results above are about, we tested every AI on two other versions. One added a hint, a line listing what to check. The other wrote the rules in, which here meant handing over, up front, every fact the relatives and restaurants would give, and his note about the card.
- What the extra help changed
- The hint didn't help, and still only 1 of 9 got everyone a seat: GPT‑6 Astra asked Auntie Mei more than once and did, and GPT‑6.1 Sol asked once and didn't. With the rules written in, 4 of 9 did, and every booking included Liu Shu's family. Even so, every try in every version failed. The best try scored 86 as written, 85 with the hint and 95 with every fact handed over.
- Who answers
- Anthropic's Sonnet 5.5 words the replies of nine of the relatives and nine of the restaurants. We wrote what each one knows, code decides which facts come out and when, and a check stops it making up figures or names. Four other relatives, one other contact and the other eight restaurants send fixed lines.
- Checking
- A script ran 88 checks on the record of bookings, messages and payments, and two AI judges from different makers read the other 5. When the judges disagreed, 11 times in 27 tries, a written ruling settled it.
- Right for the right reason
- Twenty‑two decisions score twice, once for being right and again only if the AI had found out the deciding fact before it acted. A lucky guess earns only the first.
- Time and cost
- The nine tries took between 5 and 36 minutes; the middle one took 18. For the seven AIs with published prices, a try cost between 48 cents and $6.23 at published per‑use prices for businesses. The best try took 44 steps, and the one that took 120 scored 6.