Gym operations app with three owner‑feedback repair rounds
ID laycode/task08 · revision 1 · results as of 30 October 2026
A gym owner asks for one app for members, trainers and the front desk, built from old records and chats, as a corporate client's staff contract begins. A simulated owner uses the app three times, and the agent repairs it after each visit.
- Checks
- 113, 1 point each
- Tested
- 7 models, one attempt each
- Best
- 86% (Opus 5.5)
- Median
- 82%
- Full solves
- 0 of 7
- Median attempt
- 35 min, 80 tool calls
One app for a gym, built from its old records and repaired after each owner visit
Unstated requirements
87 checks rest on facts found only in the files, such as trainer deals, and the other 26 on basics, such as moving a booking.
Clarifying with people
A simulated owner uses the app after the build and the first two repairs, sending at most three short concerns each time.
Agentic coding
113 checks on the last finished round, from payments and a 30‑second entry code to restarts, restores and 24 security checks.
Catching errors
Old records are wrong or duplicated: balances must be rebuilt from dated events, and strangers with the same name kept apart.
Disclosure and honesty
The corporate client's HR may see only what its contract allows, and a member's health condition goes only to the coach.
84 files, a payment test service and three owner visits
The agent's world
A fresh container with the owner's 84 files, read‑only, and a local payment test service, worked through the model's usual coding agent.
Owner visits
A model plays the owner: three browser visits of up to 10 minutes, after the build and two repairs, each ending in a short message.
Grading
An operator model works the final app in 12 fresh sessions, moving the clock and bringing new files, such as three bad backups.
The reward
113 equal checks on the last finished round, decided mostly by scripts and two AI judges; features that are only described don't pass.
Five apps scored 71–86%, one 12%, and one attempt handed in nothing
Each dot is one attempt, and the line marks the median.
| Model | Reward |
|---|---|
| Opus 5.5Anthropic | 86% |
| GPT‑6.1 SolOpenAI | 84% |
| GPT‑6 AstraOpenAI | 82% |
| Sonnet 5.5Anthropic | 82% |
| Fable 5.1Anthropic | 71% |
| GPT‑6 LunaOpenAI | 12% |
| Sonnet 5Anthropic | 0% |
With an IT consultant's 33 requirements added to the request, the mean was 59%, against 60% with the request alone.
They can't move a booking, take in bad backups, and mix up old records
Unstated requirements
None of the six apps could move an existing booking under the same clash and opening‑hours rules.
Five of the six got the deductions on the corporate client's invoice wrong, for equipment out of order or overdue servicing; three made none.
Agentic coding
Five of the six didn't refuse a damaged or incompatible backup while keeping the data, though all but one restored a good one.
Five of the six let a member who was already inside enter again.
Catching errors
Five of the six didn't show every imported member with their plan, dates, freezes and coach, and four mishandled repeat rows or real identity conflicts.
What an attempt costs, and what the environment includes
Cost
The median attempt cost about $8.80 at list prices for the agent model; the owner's visits and the grading models cost extra.
What's in it
The owner's 84 files, the payment test service, the task in three versions, the owner and operator prompts, the full grader, and per‑check results.
Format
Packaged as a Harbor task on request; it runs in Docker, and needs models for the owner, the operator and the judges.
Fresh variants
Other files, people and facts in the same design, made on request and never published.
The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.
Try one of our three sample environments
Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.
AsapiThe AI Agent Trials