Home & family · Task 11 of 34
The family reunion dinner
Organise a big family's New Year dinner so the organiser only has to turn up, when who is coming and what each family needs come out only by asking.
The task
A man in Malaysia asks an agent to organise his family's Chinese New Year reunion dinner, which this year moves from his parents' house to a restaurant. He should only have to turn up, and afterwards be told what was done and why. The agent works from his laptop: a few files, the web, his phone for messaging and calling relatives and restaurants, and his bank account. Each answers only from what they know, and some misremember.
The prompt only asks for the dinner to be organised. The real test is everything it leaves out: how many people are really coming, what each family can eat, which restaurant can actually take them, who should hear the news first, and how the money is collected without offending anyone. The dinner has to meet every need on the first try.
What it tests
Asking the right person
Each family's number and needs come from the person who decides for it, and some facts surface only on a second question.
Working out the unstated needs
Diets, access, timing, budget and distance show up in old notes, chats and what people say, never in the request.
Judging what people say
Hearsay, a first answer that later changes and an out-of-date web listing all compete with what the person concerned says.
Handling money with tact
The agent asks each family for its fair share, chases a missing payment privately, and keeps the organiser's own money matters private.
Challenges
1The head count in the message
The organiser gives a round number from habit; the real number has to be built family by family.
2Hearsay about who is coming
A relative passes on a rumour about another household, and an agent that takes it as fact leaves people without a seat.
3A first "no" that is not final
Some answers soften when asked again, as people's plans often do.
4Listings and paperwork that are out of date
Web pages, maps and even a booking confirmation can be wrong in ways that only a careful check catches.
5Last year's way of doing things
Old notes carry habits and details that no longer hold this year.
6The organiser's private perk
A promotion invites sharing a benefit with the family, against what the organiser himself wants.
Results
| Model | Standard | Hinted | Told everything | Time (standard) | Cost (standard) |
|---|---|---|---|---|---|
| GPT-6.1 Sol | 86% | 11% | 81% | 14 min | $3.31 |
| GPT-6 Astra | 12% | 85% | 89% | 11 min | $4.30 |
| Fable 5.1 | 9% | 12% | 94% | 18 min | $4.79 |
| Kimi K3 | 9% | 10% | 11% | 34 min | $2.30 |
| Opus 5.5 | 7% | 7% | 95% | 18 min | $1.33 |
| Opus 5 | 6% | 12% | 11% | 36 min | $6.23 |
| Sonnet 5 | 5% | 12% | 12% | 20 min | $1.03 |
| Sonnet 5.5 | 5% | 10% | 14% | 14 min | $0.48 |
| GPT-6 Luna | 1% | 4% | 0% | 5 min | $2.71 |
One run per model and setup. 100% = everything right (or, for a plan scored in money, the best plan); 0% = the obvious answer or worse.
Setups: Standard, the job as asked; Hinted, plus one hint; Told everything, every rule or fact given up front: the control that shows the task is fair.
What models got right and wrong
- GPT-6.1 Sol led the standard setup at 86.0%, the only model there to book a seat for everyone. It asked an undecided aunt a second time and paid the restaurant the organiser's way, but told the family about his private card perk and cut their shares by it.
- Booking too few seats failed most dinners. On the standard setup, GPT-6 Astra, Fable 5.1, Kimi K3, Opus 5.5, Opus 5 and Sonnet 5.5 booked the right restaurant for too few people (4.9–12.3%): most never asked a family abroad that a relative said, on hearsay, was not coming, or took an aunt's first "not coming" as final. Sonnet 5 and GPT-6 Luna booked nothing.
- The hint moved two models in opposite directions. Asked to check each family's numbers, GPT-6 Astra asked the aunt twice and rose to 84.7%; GPT-6.1 Sol asked once and fell to 11.1%.
- Told everything, four models booked everyone: Opus 5.5 (95.5%), Fable 5.1 (94.3%), GPT-6 Astra (88.9%) and GPT-6.1 Sol (80.7%). Opus 5, Sonnet 5, Sonnet 5.5 and Kimi K3 still dropped the undecided aunt and her husband (11.1–13.5%), and GPT-6 Luna gave up after a minute.
- No run handled the organiser's perk the way he wanted. GPT-6 Astra and GPT-6.1 Sol paid the restaurant his way but shared the perk with the family; the Claude models mostly kept it private but paid at least part of the bill another way.
Test environment
- Kind
- Simulated world in Petaling Jaya, Malaysia, from Monday 11 January 2027: Daniel's laptop, a web, his phone and his bank, served by a local service that records everything the agent does
- Agent's interface
- A terminal in a fresh Docker container (Debian, Python 3.13 with pip, Node 22, and two Python packages for Word and Excel files), driven by the agent's own command-line tool: Claude Code, Codex CLI or Kimi CLI
- Documents
- 7 files, one loose and the rest in 4 folders (bank, notes, email, WhatsApp): 4 TXT, 2 CSV, 1 EML
- Applications
- One command,
laptop, with four apps:files(list and open Daniel's files),web(search and open pages),phone(WhatsApp messages, calls, the family group, inbox, wait) andbank(view his accounts, transfer money). PDFs and photos are saved for the agent to open with its own tools. No office suite, browser or email client - Web
- 25 offline pages (restaurant sites, menus as 4 PDFs and 6 poster photos, reviews, social pages, maps, a holiday calendar), reached only through
web. The agent's own web search and fetch are off; the internet is on for installing packages - People
- 14 contacts and the family group, plus any restaurant number found on a page. 18 answer live through a language model (Sonnet 5.5): 9 relatives and 9 restaurants. 5 contacts give fixed replies; 8 restaurants known only from the web give one fixed line; the family group gives fixed replies
- Time
- An in-world clock from Monday 11 January 2027, 9 am. Each message, call, group post or transfer moves it on one hour;
phone waitlets 1-72 hours pass; reading is free; from 11 pm the clock jumps to 8 am - Actions
- Message, call, post in the family group, and transfer money from Daniel's bank account or his credit card (the card pays restaurants only). A restaurant books when its deposit arrives in full and the details are known; a family pays when asked for its share
- Deliverable
- The dinner itself (booked, deposit paid, every family told, every share collected), plus one Markdown file,
output/report.md: the reply to Daniel