AsapiYour AI personal assistant
Ten jobs from a busy life that you might hand to an assistant. In six, the AI works for one person: three times for Daniel Lim, through his laptop and phone, twice for a student, Jordan, on the university's registration site, and once for Dan Ashgrove, whose family is stranded abroad. In the other four, it works at a firm that does the job for people: three at a bill‑checking service, one at a travel desk. It can usually contact the people involved.
10 jobs · 11 AIs tested. GPT‑6.1 Sol did best on average, then Opus 5.5 and GPT‑6 Astra.
The ten jobs
Every AI on every job, each a dot from 0 to 100. Open a job to see what the best AI got right and wrong.
Organise the family's New Year dinner (9 AIs tested)
Daniel Lim, in Malaysia, wants his family's Chinese New Year reunion dinner, about 40 people from seven households, moved from home to a restaurant, so all he has to do is turn up.
What the best AI did
- Found out who was really coming and booked the one restaurant that fits everyone.
- Told the family about the cashback on Daniel's card, which he keeps for himself, and cut every share by it.
How each AI scored
Rescue a cancelled trip (8 AIs tested)
The Calders' flight to a family wedding near Lisbon is cancelled, and the AI, at their travel firm, has to get all four there by Friday, spending up to $1,500 without asking.
What the best AI did
- Got all four to the wedding on every try.
- Lost the airline's free seats while it checked, which left the family $680 out of pocket.
How each AI scored
Check a household's monthly bills (8 AIs tested)
At a bill‑checking firm, the AI reviews Dana Ferreira's bills for the first time since she moved house, disputes the wrong charges and writes the letters.
What the best AI did
- Got nearly every charge right, and caught a missing refund Dana hadn't noticed.
- Asked the power company, in two of three tries, for $56.40 it had already credited.
How each AI scored
Take on a new client's bills (8 AIs tested)
The same firm takes on Celia Marsh. Three of her charges need her answer, like whether she'd sent some boots back, and the AI can message her, though it isn't told it can.
What the best AI did
- Was right about every charge it could check in Celia's records.
- Kept its questions for Celia until the end, so $168 for boots she'd returned went unclaimed.
How each AI scored
Sort out a couple's medical bills (8 AIs tested)
For the same firm, the AI goes through a year of Dennis and Joanna Vellacott's medical bills across two health plans, writing to the plans, the doctors and the office at Dennis's work that runs his plan.
What the best AI did
- Applied the insurance rules correctly on every try.
- Described Dennis's medical care in its letters to the office at his work.
How each AI scored
Decide on solar panels (10 AIs tested)
Daniel wants the best solar plan for his home, and a fair deal with his brother Kenneth, who covers the power bill in place of rent.
What the best AI did
- Chose the right system, sorted out the roof and found the best price.
- Told Daniel to wait and pay cash, without saying that waiting loses a RM 3,000 rebate.
How each AI scored
Plan an emergency fund (8 AIs tested)
Daniel wants an emergency fund worth six months of his family's costs, and then RM 20,000 on top. He's too busy for questions, but his wife, his family and HR can answer them.
What the best AI did
- Made the strongest plan, after asking four people about costs, dates and pay that had changed.
- Moved RM 4,860 of other card debt onto the card meant for emergencies.
How each AI scored
Sort out next term's classes (4 AIs tested)
A student, Jordan Avery, needs next term's classes to work around library shifts, swimming and a time‑limited parking permit, with enough credits to keep a scholarship.
What the best AIs did
- Built a timetable that fits around Jordan's library shifts, swimming and parking permit.
- Skipped the note from Jordan's advisor and missed the course it asks for.
How each AI scored
Sort out classes and graduate sooner (1 AI tested)
Jordan also wants the quickest route to a degree, and three courses compete for one slot in the week.
What the best AI did
- Read the note from Jordan's advisor and closed three requirements, as it asks.
- Picked a Spanish class Jordan can't take yet, and can't reach in time after swimming.
How each AI scored
Bring a stranded family home (8 AIs tested)
Marina's flight home from Brazil to Boston with the two children and their dog is cancelled, and the AI, as her husband Dan Ashgrove's personal assistant, has to get the family home by Wednesday night, the day his parents arrive for Thanksgiving.
What the best AI did
- Got Marina and the children home by Wednesday on every try, and left the dog with their grandmother, as the family had agreed.
- On every try, it wrote every to‑do, and most times and bookings, differently from the assistant's past notes.
How each AI scored
Each AI had three tries at the two trips and the three bills jobs, so we could see how steady it was.
Who did best
Anthropic's Opus 5.5 had the best result on four of the ten jobs, one of them shared, and Opus 5 on getting a stranded family home. OpenAI's GPT‑6.1 Sol, a mid‑sized AI, had the best on two, and was the only AI to seat everyone at the dinner. OpenAI's GPT‑6 Luna, a small AI we tested on eight of the jobs, did far worse: on the three bills jobs, seven of its nine tries ended in under 90 seconds, with most of the job undone.
Average score out of 100 over the jobs each AI was tested on, for the 7 AIs tested on more than half of the ten jobs.
What the ten jobs showed
What they do well
They can work through a firm's records and rules.
At the bill‑checking firm, the AI works in a system of 82,000 to 98,000 records. Anthropic's Opus 5.5 spotted things Dana had missed, like a refund a bookshop never paid, and Fable 5.1 applied the rules of Dennis and Joanna's two health plans correctly in every try.
They can plan around a lot of fixed commitments.
Three AIs fitted Jordan's classes around a job, noon swim training and a six‑hour parking limit, with time for every walk. On the wedding trip, all four Calders would have reached the wedding in time in 21 of the 24 tries.
Where they fall short
They rarely ask the people who know, unless the request points them that way.
On four of the jobs, someone could have told the AI what it got wrong, and most AIs never asked. Only one of the ten AIs on the solar plan wrote to Daniel's brother, Kenneth, and all 24 tries on Celia's bills finished without messaging her. Told in one sentence that she could be messaged, the three AIs we tested it on wrote to her in all five of their tries.
They miss how a firm likes things done when only its old letters show it.
The bill‑checking firm writes its letters a set way, and only its past letters show how. On each of the three bills jobs, missing those habits cost nearly every AI more than anything else. When we wrote them into the request for the medical bills, the better of the two AIs we tested failed on just one of the 351 things we checked.
How these jobs were set up and scored
- The worlds
- Each job runs in a world we made up, as realistic as we could make it. It might be a laptop full of files and apps, a university's registration site, a bill‑checking firm's system of 82,000 to 98,000 records, a travel desk with airline, hotel and fare sites, or a family's mail, messages and booking sites. The AI only sees what it opens.
- The people
- An AI words the replies of most relatives, the clients whose bills are being checked, the airline agent and HR, from what we wrote for each of them. Code decides which facts each one may give, and only when asked the right question. Other contacts send fixed replies.
- Other versions of the request
- The results above are for the request as a person would write it. On three jobs, we also tested every AI on a version with a one‑line hint, and one with every rule the job depends on written in. On the dinner, that version also handed over every fact the relatives would give. On four more jobs, we tested one to three AIs on one of these versions.
- With the rules written in
- Given every rule up front, three AIs got everything right on the solar plan, the only tries on these ten jobs that did. On the two trips and the medical bills, we tested two AIs on it, and each time the better of the two missed one check.
- Scores
- A score is the share of a job's checks an AI got right, with the important ones counting more, and 100 means everything right. Where an AI had three tries, the score is the average, and it only counts as getting a job right if every try did. On the dinner, ten checks are make‑or‑break. Miss one, like a guest without a seat, and most of the score goes.
- Time and cost
- Most tries took 5 to 30 minutes. For the AIs with published prices, a try would cost from 7 cents to $9.46 on the five jobs where every AI's cost was recorded.
Every AI's score on every job
| AI | Dinner | Trip | Bills | New client | Medical | Solar | Savings | Classes | Graduate sooner | Bring a stranded family home |
|---|---|---|---|---|---|---|---|---|---|---|
| Opus 5Anthropic | 6 | 66 | 86 | 79 | 67 | 54 | 61 | 79 | 79 | 92 |
| Fable 5.1Anthropic | 9 | 69 | 88 | 80 | 86 | 74 | 86 | 79 | – | 56 |
| Opus 5.5Anthropic | 7 | 81 | 91 | 79 | 80 | 72 | 89 | 79 | – | 91 |
| GPT‑6 AstraOpenAI | 12 | 71 | 89 | 82 | 81 | 54 | 83 | – | – | 90 |
| GPT‑6 LunaOpenAI | 1 | 15 | 31 | 26 | 5 | 38 | 51 | – | – | 30 |
| GPT‑6.1 SolOpenAI | 86 | 70 | 88 | 80 | 77 | 78 | 83 | – | – | 72 |
| Sonnet 5.5Anthropic | 5 | 73 | 84 | 70 | 70 | 58 | 64 | – | – | 35 |
| GPT‑6 SolOpenAI | – | 62 | 78 | 73 | 68 | – | – | – | – | 36 |
| Sonnet 5Anthropic | 5 | – | – | – | – | 39 | 48 | 57 | – | – |
| Kimi K3 openMoonshot | 9 | – | – | – | – | 71 | – | – | – | – |
| GLM‑5.3‑Flash openZhipu | – | – | – | – | – | 51 | – | – | – | – |
These are the scores out of 100 for the request as a person would write it, averaged where an AI had three tries. Darker means higher. A dash means we didn't test that AI on that job. Open marks an AI anyone can download.