AsapiThe AI Agent Trials
Asapi / Catalogue / Music‑teaching studio app: lessons, billing, media
Environment datasheet · Small business

Music‑teaching studio app: lessons, billing, media

ID laycode/task05 · revision 1 · tested 19–20 October 2026

A self‑employed piano teacher asks for one web app for her studio: lessons, pupils' work, recordings, sign‑ins, invoices, online payment and tax records. The agent works out prices, terms and consent from her files, builds it and leaves a note.

Checks
155–179, in three weighted parts
Tested
7 models, one attempt each
Best
81% (Sonnet 5.5)
Median
76%
Full solves
0 of 7
Median attempt
20 min, 55 tool calls
What it tests

She names the features; her files hold the rules behind them

Unstated requirements

Price changes, family billing, cancellation terms and recording consent are only in 416 files that don't all agree; basics like backups count too.

Agentic coding

The app must take test payments confirmed by the payment service, record audio and video in lessons, and pass 30 security checks.

How it works

416 files, a test payment service and five months

The agent's world

A fresh container with the teacher's 416 files and a test payment service, worked through the model's usual coding agent.

Limits

Two hours, 150 model turns and a $20 spend cap at list prices, with web search off.

Grading

A tester model uses the app as teacher, pupils and parents with a fake camera and microphone; two AI judges score what it saw.

The reward

155–179 checks, fewer where an app lacks a feature; the app's features count 60%, edge cases 20% and security 20%.

Results

Most attempts scored 75–81%, and two fell far below

Each dot is one attempt, and the line marks the median.

MedianFully solved
7 modelsOne attempt each76%0 of 7
ModelReward
Sonnet 5.5Anthropic81%
Opus 5.5Anthropic80%
Fable 5.1Anthropic77%
GPT‑6.1 SolOpenAI76%
GPT‑6 AstraOpenAI75%
Sonnet 5Anthropic35%
GPT‑6 LunaOpenAI26%

With an IT consultant's 33 requirements added to the request, the best three scored 93–94%.

What models get wrong

They skip recovery, ignore withdrawn consent, and keep departed families' logins open

Unstated requirements

  1. Only 14% of apps could restore a backup and refuse a damaged one, and no note said both where the data and backups are kept and how to make a backup.

  2. Only one app, GPT‑6 Astra's, blocked new video recording and sharing once a family withdrew consent.

Agentic coding

  1. After handover, only 14% of apps could take in new bank files and parents' messages through their own pages.

  2. In most apps, a departed pupil's family kept its billing login after the final bill was paid.

Running it

What an attempt costs, and what the environment includes

Cost

The median attempt cost about $7 at list prices for the agent model; the tester's and judges' calls are extra.

What's in it

Her files, the test payment service, the task in three versions from the request alone to a consultant's requirements, the full grader, and per‑check results.

Format

Packaged as a Harbor task on request; grading needs Docker, a browser, and models for the tester and judges.

Fresh variants

Other files, people and facts in the same design, made on request and never published.

The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.