AsapiThe AI Agent Trials
Asapi / Catalogue / Gym operations app with three owner‑feedback repair rounds
Environment datasheet · Small business

Gym operations app with three owner‑feedback repair rounds

ID laycode/task08 · revision 1 · results as of 30 October 2026

A gym owner asks for one app for members, trainers and the front desk, built from old records and chats, as a corporate client's staff contract begins. A simulated owner uses the app three times, and the agent repairs it after each visit.

Checks
113, 1 point each
Tested
7 models, one attempt each
Best
86% (Opus 5.5)
Median
82%
Full solves
0 of 7
Median attempt
35 min, 80 tool calls
What it tests

One app for a gym, built from its old records and repaired after each owner visit

Unstated requirements

87 checks rest on facts found only in the files, such as trainer deals, and the other 26 on basics, such as moving a booking.

Clarifying with people

A simulated owner uses the app after the build and the first two repairs, sending at most three short concerns each time.

Agentic coding

113 checks on the last finished round, from payments and a 30‑second entry code to restarts, restores and 24 security checks.

Catching errors

Old records are wrong or duplicated: balances must be rebuilt from dated events, and strangers with the same name kept apart.

Disclosure and honesty

The corporate client's HR may see only what its contract allows, and a member's health condition goes only to the coach.

How it works

84 files, a payment test service and three owner visits

The agent's world

A fresh container with the owner's 84 files, read‑only, and a local payment test service, worked through the model's usual coding agent.

Owner visits

A model plays the owner: three browser visits of up to 10 minutes, after the build and two repairs, each ending in a short message.

Grading

An operator model works the final app in 12 fresh sessions, moving the clock and bringing new files, such as three bad backups.

The reward

113 equal checks on the last finished round, decided mostly by scripts and two AI judges; features that are only described don't pass.

Results

Five apps scored 71–86%, one 12%, and one attempt handed in nothing

Each dot is one attempt, and the line marks the median.

MedianFully solved
7 modelsOne attempt each82%0 of 7
ModelReward
Opus 5.5Anthropic86%
GPT‑6.1 SolOpenAI84%
GPT‑6 AstraOpenAI82%
Sonnet 5.5Anthropic82%
Fable 5.1Anthropic71%
GPT‑6 LunaOpenAI12%
Sonnet 5Anthropic0%

With an IT consultant's 33 requirements added to the request, the mean was 59%, against 60% with the request alone.

What models get wrong

They can't move a booking, take in bad backups, and mix up old records

Unstated requirements

  1. None of the six apps could move an existing booking under the same clash and opening‑hours rules.

  2. Five of the six got the deductions on the corporate client's invoice wrong, for equipment out of order or overdue servicing; three made none.

Agentic coding

  1. Five of the six didn't refuse a damaged or incompatible backup while keeping the data, though all but one restored a good one.

  2. Five of the six let a member who was already inside enter again.

Catching errors

  1. Five of the six didn't show every imported member with their plan, dates, freezes and coach, and four mishandled repeat rows or real identity conflicts.

Running it

What an attempt costs, and what the environment includes

Cost

The median attempt cost about $8.80 at list prices for the agent model; the owner's visits and the grading models cost extra.

What's in it

The owner's 84 files, the payment test service, the task in three versions, the owner and operator prompts, the full grader, and per‑check results.

Format

Packaged as a Harbor task on request; it runs in Docker, and needs models for the owner, the operator and the judges.

Fresh variants

Other files, people and facts in the same design, made on request and never published.

The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.