Home & family · Task 7 of 7
A family piano practice app
Build one web app that a piano family of three levels practises with at home, when it must listen to a real piano and every detail sits in their notes.
The task
A mother in a family of three pianists asks for one web app that all three use at home on one laptop. She names the main features: guided lessons, a duet partner and an orchestra that follow the player, feedback on mistakes, transcription of piano recordings, speed and hand practice, and private logins. The agent gets her computer's files: scores, MIDI copies, orchestra recordings, a catalogue, the teacher's practice guide and the family's notes and messages. Nobody answers questions.
The prompt only asks for the features she names; the real test is whether they work for a real player. A careful developer notices that the app listens to an acoustic piano through a microphone, that the teacher's guide sets how practice is scored and paced, and that each family member's practice is private.
What it tests
Finding unstated needs
Work out from the family's notes and the teacher's guide what each feature must do without being told.
Reading audio and scores
Use scores, MIDI files and recordings that can only be checked by listening to them or measuring them.
Weighing sources
Follow the teacher's rules over the agent's own idea of how practice should work.
Challenges
1A real piano, not a keyboard
The app must hear chords from an acoustic piano through a laptop microphone.
2Real sound, real lag
Sound from the room reaches the app late, as it does on any laptop.
3Following the player
The partner and the orchestra must keep time with the player, not lead.
4The teacher's rules
The practice guide sets the scoring and pacing in a teacher's plain words.
5Private practice
Each family member keeps their own history and their own choice about recording.
Results
| Model | Standard | Hinted | Told everything | Time (standard) | Cost (standard) |
|---|---|---|---|---|---|
| GPT-6 Astra | 58% | 51% | 52% | 41 min | $11.19 |
| Sonnet 5.5 | 56% | 59% | 60% | 43 min | $5.68 |
| Opus 5.5 | 55% | 60% | 52% | 64 min | $14.88 |
| Fable 5.1 | 53% | 49% | 62% | 53 min | $16.57 |
| GPT-6.1 Sol | 49% | 46% | 49% | 22 min | $0.89 |
| Sonnet 5 | 34% | 29% | 34% | 39 min | $8.12 |
| GPT-6 Luna | 20% | 21% | 29% | 3 min | $2.35 |
One run per model and setup. 100% = everything right (or, for a plan scored in money, the best plan); 0% = the obvious answer or worse.
Setups: Standard, the job as asked; Hinted, plus one hint; Told everything, every rule or fact given up front: the control that shows the task is fair.
What models got right and wrong
- GPT-6 Astra led the standard setup at 57.7%, with Sonnet 5.5 (55.8%), Opus 5.5 (55.0%) and Fable 5.1 (53.4%) close behind. No run reached 62%.
- No app allowed for the late sound, so a clean take showed mistakes in all 21 runs of GPT-6 Astra, Sonnet 5.5, Opus 5.5, Fable 5.1, GPT-6.1 Sol, Sonnet 5 and GPT-6 Luna. Five runs saw the lateness in their own tests and handed over without a fix.
- In no run did the duet partner or the orchestra keep time with the player; the agents tested following only against what their own app heard.
- 20 of 21 runs built their own speed ladder instead of the teacher's.
- GPT-6 Luna stayed under 30% in every setup, and Sonnet 5 under 35%.
- More guidance helped some models and cost others: the told-everything setup raised GPT-6 Luna 9.8 points and Fable 5.1 8.1, and lowered GPT-6 Astra 5.5 and Opus 5.5 2.8.
Test environment
- Kind
- Simulated family at home: the owner's computer is a folder of scores, recordings, family notes and the piano teacher's practice guide; the agent builds a working web app
- Agent's interface
- A terminal in a fresh Debian Docker container, driven by Claude Code or Codex CLI; Python 3.13, Node 22, PyMuPDF, pdfplumber, Pillow, Playwright with Chromium, Ghostscript and Tesseract are available
- Documents
- 28 files, read-only at
/home/user/hana/: eight TXT, seven MusicXML, seven MIDI, four WAV, one CSV and one JSON file, in four folders:Music/(21, of which eight are inHarbour Morning - Backing/),Piano lessons/(3),Home/(2) andMessages/(2) - Applications
- None during the build. During grading, the app runs in headed Chromium with a virtual speaker and a virtual microphone; the grader plays recorded piano takes into the microphone. In this test browser, a perfect take reaches the app 0.32–0.36 s after the click it was played to. The browser's reported latency numbers cover only 0.27–0.29 s of that delay
- Web
- None. Internet access is available for package installation; the agents' own web search and fetch tools are off
- People
- Nobody answers. Hana, Marek, Ela and their teacher Clara appear only in saved notes, messages and the practice guide
- Time
- The prompt sets the task on Wednesday 21 October 2026, 08:00 CEST in Brno. The containers' clocks are not changed
- Actions
- Read the owner's files; write and run code in the workspace; nothing outside the container
- Deliverable
output/app/withoutput/app/start.sh, serving on port 8080, and a short note atoutput/NOTE.md