AsapiYour AI hospital assistant
We built three jobs for made‑up hospitals and health services. In one, the AI coordinates a ward for three days and nights. It turns doctors' orders into tasks for the nurses and the test department, and calls a doctor when a patient gets worse. In the other two, it works alone from a manual thousands of lines long to build a program hospital staff would rely on. One keeps patients' radiation‑dose records, and the other gathers an emergency patient's history.
3 jobs · 13 AIs tested. Opus 5.5 did best on average, then Fable 5.1 and GPT‑6.1 Sol.
The three jobs
Every AI on every job, each a dot from 0 to 100. Open a job to see what the best AI got right and wrong.
Coordinate a hospital ward for three days (6 AIs tested)
For three days and nights, the AI makes sure the doctors' orders on a ward get done.
What the best AI did
- Kept every dose on time, followed up every test, and paged a doctor only when needed.
How each AI scored
Build a radiation‑dose records program (12 AIs tested)
The AI builds the program that keeps each patient's radiation‑dose record, exactly as a 2,448‑line manual describes.
What the best AIs did
- Built the program with all 466 checks right, on records they had never seen.
How each AI scored
Build an emergency department's patient summary (6 AIs tested)
The AI builds the program that gathers everything known about a patient arriving at an emergency department, as a 4,555‑line manual describes.
What the best AIs did
- Built the program with all 919 checks right, including rules that survive only in forms an old system filled in.
How each AI scored
Who did best
Anthropic's Opus 5.5 got everything right on all three jobs, and was the only AI to run the ward without a miss. On the dose‑records program, the eight that got everything right included OpenAI's GPT‑6 Astra and GPT‑6.1 Sol and two AIs anyone can download, DeepSeek V4.1 Flash and Qwen3.8‑27B. On the emergency summary, Sonnet 5.5 got everything right too, and Fable 5.1 and GPT‑6.1 Sol missed one check each.
Average score out of 100 over the jobs each AI was tested on, for the 7 AIs tested on more than half of the three jobs.
What the three jobs showed
What they do well
Given a manual, they can build programs exactly right.
Building the dose‑records program from a 2,448‑line manual, eight of twelve AIs got all 466 checks right on records they had never seen, two of them AIs anyone can download. On the emergency summary, Opus 5.5 and Sonnet 5.5 got all 919 right, including rules kept only in forms an old system filled in.
The strongest can run a ward for three days.
On the ward, Opus 5.5 kept every dose on time, followed up every test and paged a doctor only when needed, over three days and nights.
Where they fall short
On the ward, most still miss something that matters.
Five of the six AIs failed the ward. GPT‑6 Astra paged doctors 98 times about one scan instead of ordering the blood test it needed, and the scan never happened.
One small AI falls far behind.
GPT‑6 Luna hadn't asked to be woken for five hours one night, and missed a patient getting worse. It scored 4 and 10 on the two programs.
How these jobs were set up and scored
- The ward
- A simulated ward run by code. The nurses, doctors and patients follow set rules, and patients' readings respond to the doses given or missed. Over 72 simulated hours the AI is woken 127 to 217 times, each time with only its notes.
- The two programs
- Each was built alone in 90 minutes from a manual and example files, then run by tests on records the AI had never seen: 466 checks for the dose records program and 919 for the emergency summary.
- Scores
- On the ward, a score is the share of its own checks the AI got right, between 285 and 410 depending on what happened on its ward. On the programs, it's the share of tests passed. One fault in a program can fail many tests at once, so a low score can be a single mistake.
- Time and cost
- A program took 6 to 90 minutes, and three days on the ward took 104 to 238 minutes of the AI's own working time. Where recorded, a try cost from about 6 cents, an estimate, to $137.67 at published per‑use prices for businesses.
Every AI's score on every job
| AI | Coordinate a hospital ward for three days | Build a radiation‑dose records program | Build an emergency department's patient summary |
|---|---|---|---|
| GPT‑6 AstraOpenAI | 73 | 100 | 99 |
| GPT‑6 LunaOpenAI | 82 | 4 | 10 |
| Opus 5.5Anthropic | 100 | 100 | 100 |
| Sonnet 5.5Anthropic | 99 | 99 | 100 |
| DeepSeek V4.1 Flash openDeepSeek | 97 | 100 | – |
| Fable 5.1Anthropic | – | 100 | 99 |
| GPT‑6.1 SolOpenAI | – | 100 | 99 |
| GLM‑5.3‑Flash openZhipu | – | 0 | – |
| GPT‑5.6 SolOpenAI | – | 100 | – |
| GPT‑6 SolOpenAI | 98 | – | – |
| Opus 4.8Anthropic | – | 98 | – |
| Opus 5Anthropic | – | 100 | – |
| Qwen3.8‑27B openAlibaba | – | 100 | – |
These are the scores out of 100 for the request as a person would write it, one try per AI. Darker means higher. A dash means we didn't test that AI on that job, and a 0 that it handed in nothing. Open marks an AI anyone can download.