AsapiThe AIs we tested
We tested 23 AIs from eight makers. They include OpenAI's GPT models, the family behind ChatGPT, Anthropic's Claude models, such as Opus, Sonnet and Fable, and one of Google's, Gemini 4 Argon. The other five makers offer AIs anyone can download. OpenAI's ran in Codex, almost all of Anthropic's in Claude Code and Google's in Antigravity, the makers' own apps for running an AI as an agent on a computer. We haven't tested the ChatGPT or Claude chat apps, or Microsoft's AIs, and we didn't test every AI on every job.
The three strongest were close
On the 50 jobs we gave all five of the most‑tested AIs, Opus 5.5, GPT‑6 Astra and GPT‑6.1 Sol came out close, well ahead of Sonnet 5.5 and GPT‑6 Luna. Even they got everything right on only 4 to 7 of those jobs.
| AI | Size | Everything right | Average score |
|---|---|---|---|
| Opus 5.5Anthropic | Large | 7 of 50 | 79 |
| GPT‑6 AstraOpenAI | Large | 5 of 50 | 78 |
| GPT‑6.1 SolOpenAI | Mid‑sized | 4 of 50 | 76 |
| Sonnet 5.5Anthropic | Mid‑sized | 2 of 50 | 68 |
| GPT‑6 LunaOpenAI | Small | 0 of 50 | 33 |
Everything right: the jobs where the AI met every check, in every try we counted. Average score: its average out of 100 across the jobs these five shared.
Who did best in each benchmark
- Your AI personal assistant
- Opus 5.5 had the best result, or tied for it, on four of the ten jobs; GPT‑6.1 Sol was the only AI to get everyone a seat at the family dinner.
- Your AI office assistant
- GPT‑6 Astra did best on the office jobs, and got everything right on two: what to do about a shop's broken ice cream freezer, and a bakery's fire claim.
- Your AI app builder
- Opus 5.5 had the best result on seven of the 13 apps, and was the only AI to get everything right on one, the train‑ticket catcher.
- Your AI customer service agent
- GPT‑6 Astra, Opus 5.5 and GPT‑6.1 Sol each got all 80 repair customers right, and Opus 5.5 had the best result, or tied for it, on four of the six jobs.
- Your AI operations manager
- Opus 5.5 got everything right on four of the eight long jobs.
- Your AI hospital assistant
- Opus 5.5 got everything right on all three jobs, and was the only AI to do so on the ward.
- Your AI facilities engineer
- Fable 5.1 did best on both jobs, and was the only AI to explain each cause in full.
- Your AI engineering assistant
- Sonnet 5.5 got everything right reading the plant's scans, and Fable 5.1 bringing its drawings up to date.
How much size matters
We sort most of the AIs into large, mid‑sized and small, as a rough guide. Bigger usually does better, but not always. On the 52 jobs where we tested AIs of all three sizes, a large AI got everything right on 11, a mid‑sized one on 6 and a small one on 1. GPT‑6.1 Sol, a mid‑sized AI, kept up with the two strongest large ones.
AIs anyone can download
Five makers offer AIs anyone can download and run. DeepSeek V4.1 Flash came closest to the strongest, and shared the top score on 2 of the 12 jobs we tested it on. It got everything right on the radiation‑dose records program. On the freezer's insurance claim it tied for closest, though no AI got everything right. It failed the hospital ward too, but missed little. Qwen3.8‑27B also got everything right on the radiation‑dose records program.
Every AI we tested
| AI | Size | Jobs tested | Everything right |
|---|---|---|---|
| Opus 5.5Anthropic | Large | 55 | 9 |
| GPT‑6 AstraOpenAI | Large | 53 | 6 |
| GPT‑6.1 SolOpenAI | Mid‑sized | 52 | 4 |
| Fable 5.1Anthropic | Large | 45 | 3 |
| Sonnet 5.5Anthropic | Mid‑sized | 53 | 2 |
| Opus 5Anthropic | Large | 31 | 1 |
| DeepSeek V4.1 Flash openDeepSeek | Small | 12 | 1 |
| Qwen3.8‑27B openAlibaba | Small | 10 | 1 |
| GPT‑6 LunaOpenAI | Small | 52 | 0 |
| Sonnet 5Anthropic | Mid‑sized | 32 | 0 |
| GPT‑6 SolOpenAI | Mid‑sized | 13 | 0 |
| GLM‑5.3‑Flash openZhipu | Small | 10 | 0 |
| Kimi K3 openMoonshot | Mid‑sized | 9 | 0 |
| GPT‑5.6 LunaOpenAI | Small | 5 | 0 |
Everything right: the jobs where the AI met every check, in every try we counted. We tested nine more AIs on one job each: DeepSeek V3.2 (DeepSeek), GLM‑5 (Zhipu), GLM‑5.3 (Zhipu), GPT‑5.6 Sol (OpenAI), Gemini 4 Argon (Google), Kimi K2.5 (Moonshot), Nemotron Super 3 (NVIDIA), Opus 4.8 (Anthropic), Qwen3‑235B (Alibaba); GPT‑5.6 Sol got everything right on its one job, the radiation‑dose records program.