AsapiYour AI DIY assistant
We built six jobs around homes. In three, the AI works for firms that furnish rooms: it buys furniture to match a client's photos, replaces a flooded family's furniture for their insurer, and gets a house ready for a family of six. In the others, it plans a homeowner's bedroom lights, fixes a wet condo floor overnight with what's in the condo, and quotes rooftop solar for a row house.
6 jobs · 10 AIs tested. Opus 5.5 did best on average, then Sonnet 5.5 and Fable 5.1.
The six jobs
Every AI on every job, each a dot from 0 to 100. Open a job to see what the best AI got right and wrong.
Furnish a room from six photos (8 AIs tested)
Dana Okafor wants the living‑dining room of her new apartment exactly like the model apartment's, which she has only in six photos, and the AI, at a furnishing service, orders every piece within $9,000 and draws the floor plan for the delivery crews.
What the best AI did
- Stayed under the $9,000 budget with nothing late every time, and once ordered all 17 pieces exactly right.
- Its floor plan put a typical piece 40 to 70 cm from where it stands in the photos.
How each AI scored
Replace a family's flooded furniture (8 AIs tested)
A burst hose flooded a family's bedroom and study while they were away, and the AI, at a firm that replaces furniture for insurers, works out from their photos what stood there, buys what the insurer will replace and plans where the crews put it.
What the best AI did
- Bought the right product in the right colour for every piece but one across its three tests, and placed more pieces where they had stood than any other AI.
- Bought the bench one size off in every test, so it doesn't match the one the family lost.
How each AI scored
Get a house ready for a family of six (8 AIs tested)
Nathan and Elise, their two children and Elise's parents, one of them three weeks out of hip surgery, are moving into a 1970s house, and the AI decides who sleeps where, buys what they still need within $7,200 and lays out every room.
What the best AI did
- Gave every room to the right people and kept to the budget and the two delivery days.
- In every test, it put furniture over six to eight heating vents, which blocks the heat. It also left furniture in front of the four‑year‑old's windows, where she could climb up to them.
How each AI scored
Plan a homeowner's bedroom lighting (8 AIs tested)
Marco, a handy homeowner in New Zealand, wants to fit six new ceiling lights in his bedroom himself, and the AI plans them from his notes, photos and one shop's catalogue and writes him a step‑by‑step guide.
What the best AIs did
- Got the wiring right throughout, including a switch by the bed that still works after the one by the door is turned off.
- Fable 5.2 told him a test he can do at home proves the wires aren't mixed up, which it can't. GPT‑6.1 Sol left a light shining in his eyes as he reads in bed, the glare he wanted gone.
How each AI scored
Fix a wet condo floor overnight (7 AIs tested)
The owner of a short‑stay condo arrives at 9 p.m., the night before a guest checks in, to a hall floor soaked by a leak, and the AI, standing in for the owner, has to stop it and rebuild the floor with what's in the condo by 1:30 a.m.
What the best AI did
- Passed every check in one of its three tests, opening the floor only as far as it was wet.
- In the other two, it opened more floor than its materials could cover, and the repair didn't end up level with the old floor.
How each AI scored
Quote rooftop solar for a row house (6 AIs tested)
A family in a street of near‑identical row houses asks a solar installer for a quote by email, and the AI finds their house in plans and overhead photos, works out the neighbours' shade, and designs and prices the panels.
What the best AIs did
- Found the family's house among look‑alikes and designed panels that earn almost all the best design could, with an honest quote.
- Their panels earn the family $4 to $7 less over ten years than the best design we found.
How each AI scored
We tested each AI three times on the three furnishing jobs and the wet floor, so we could see how steady it was.
Who did best
Anthropic's Opus 5.5 had the best average, with the top score on the flood claim and the family's move, and it shared the top score on the rooftop solar with Fable 5.2. OpenAI's GPT‑6 Astra did best on the furnished room and the wet floor, but declined to design the rooftop solar at all. GPT‑6.1 Sol and Fable 5.2 shared the top score on the lighting plan. GPT‑6 Luna, a small AI, came last on five of the six jobs.
Average score out of 100 over the jobs each AI was tested on, for the 6 AIs tested on at least four of the six jobs.
What the six jobs showed
What they do well
They get the big decisions right.
Getting the family's house ready, 19 of the 23 plans the AIs handed in gave every room to the right people, and Opus 5.5 did it in all three tests, within the $7,200 budget and on the two delivery days. On the rooftop solar, four of six AIs found the family's house among look‑alikes.
Given the facts in writing, they get nearly everything right.
When we added the family's order confirmations, Opus 5.5 and GPT‑6.1 Sol bought every piece exactly right. With a measured plan and the move manager's rules for the house, they laid out every room right, and with the method added, all 20 tests fixed the wet floor.
Where they fall short
They misjudge sizes and shades in photos.
Replacing the flooded furniture, 23 of 24 tests didn't buy the bench the family lost, and 18 didn't buy their loveseat, most often a shade or a size off, which leaves the family with pieces that don't match what they had.
They miss how people will live in the room.
Getting the family's house ready, 22 of the 23 plans put furniture over a heating vent or against the heater, which blocks the heat. On the lighting plan, 6 of 8 AIs placed a light that shines in the homeowner's eyes as he reads in bed, the glare he wanted gone.
How these jobs were set up and scored
- The jobs
- The three furnishing jobs run on three furniture shops' websites, from photos and, on two of them, plans and the clients' papers. The lighting plan works from a homeowner's notes, nine photos and one shop's catalogue, and he can be messaged. The wet floor is a simulated condo, worked through typed commands. The solar quote uses the installer's documents and five websites, among them the city's plan register and the electricity company's.
- Scores
- Most scores are the share of a job's checks met, with the important ones counting more on the furnishing jobs, and 100 means everything right. The solar quote is scored half on its checks and half on how much money the design makes the family over ten years, compared with the best design we found.
- Versions of the request
- The scores here are for the request as a person would write it. With the facts or the method written into the request, the AIs got everything right in 27 of 34 tests, against 1 of 107 with the request alone.
- Time and cost
- A test took from under a minute to about an hour and 40 minutes; the middle one took 14 minutes. On the lighting plan, the wet floor and the rooftop solar, where every AI's cost was recorded, a test cost from 4 cents to $18.40 at published per‑use prices for businesses.
Every AI's score on every job
| AI | Furnish a room from six photos | Replace a family's flooded furniture | Get a house ready for a family of six | Plan a homeowner's bedroom lighting | Fix a wet condo floor overnight | Quote rooftop solar for a row house |
|---|---|---|---|---|---|---|
| GPT‑6 AstraOpenAI | 74 | 81 | 88 | 92 | 77 | 3 |
| GPT‑6 LunaOpenAI | 8 | 19 | 37 | 69 | 41 | 37 |
| GPT‑6.1 SolOpenAI | 70 | 77 | 86 | 96 | 56 | 47 |
| Opus 5.5Anthropic | 73 | 82 | 90 | 92 | 54 | 99 |
| Sonnet 5.5Anthropic | 62 | 81 | 81 | 88 | 49 | 95 |
| Fable 5.1Anthropic | 53 | 64 | 82 | 92 | 72 | – |
| Fable 5.2Anthropic | – | – | – | 96 | 56 | 99 |
| GPT‑6 SolOpenAI | 45 | 67 | 70 | – | – | – |
| Opus 5Anthropic | 48 | 74 | 82 | – | – | – |
| Sonnet 5Anthropic | – | – | – | 88 | – | – |
These are the scores out of 100 for the request as a person would write it, the average of three tests on the furnishing jobs and the wet floor and one test on the others. Darker means higher. A dash means we didn't test that AI on that job.