Inferring unstated requirements

No file states them; past work shows them

Past examples

For the waste and price reports, reactions to past reports, chats and handwriting on a scan and a whiteboard set what 23 and 28 checks test. A firm's note and letter habits show only in its past notes, letters, review packs or closed cases.

Weights

House style carries 30% of the flight rebooking's reward, 16% in two bill reviews and 12% in a third, 10% in the stranded‑family rebooking; in the two reports, the other house requirements carry 15%.

The owner's files

In four of the first seven app builds the request is a few lay sentences, which a script confirms avoid the checks' subjects; in three it lists features. In all twelve, needs sit in the owner's files or, for train tickets, the site's pages, beside unasked basics such as backups.

Rules in a portal

In the two course registrations, a work shift, a training slot, a parking permit, scholarship terms, degree rules and an advisor's note sit across the student's portal pages; no page states a rule as a rule.

Practice over the manual

At the spares stores, the manuals' parts lists are right on 12 of 33 counted reports; what repairs really take shows in past work. The repair desk's past jobs show parts the manuals under‑list; the tea factory's practice sits only in its chat.

Standing needs

In the holiday‑rental chat, each customer's lasting needs sit in a profile of 45 to 49 dated entries, some later corrected; messages never restate them or say how they relate to the last one.

The latest word

Files disagree over time: the owner's latest rule against last month's pack, a director's objection against the old template, a firm's latest letters against an older template on file.

Rules left in forms

In the patient‑history compiler, five of nine kinds of answer are documented nowhere but in 22 forms an old system filled; hidden tests compare every field of each form the program fills.

Checked one by one

Each requirement is a check of its own, often field by field: register cells by script, each pick‑list entry exactly, each habit per instance in a note.

Found in the world

In the server‑room incident, each of eight people holds part of the authority and the facts, and the brief gives no limits; the cooling diagnosis names no method; in the stranded‑family rebooking, a household's four note conventions show only in past summaries.

They follow the template and miss the newer word

Each count covers every model tested on the environment, with only the request; where a model made several attempts, the line says so.

1. Last month's pack over the owner's word. Bakery wholesale pack: 12 of 13 attempts, one per model, kept in the totals an account the owner had said, once in a staff chat, stays out of every total; only GPT‑6.1 Sol left it out.

2. The old template followed. Brewery input‑price report: 9 of 12 attempts, one per model, gave a verdict on last month's recommendations, as the outdated template asks, though the director had since objected to it.

3. A profile read, not applied. Multi‑turn holiday‑rental search chat: 4 of 6 attempts read the first customers' profiles in full, then searched without the lasting needs they hold, such as a lift. Once a profile update made Opus 5.5 apply them, it got all 17 remaining messages right.

4. The firm's habits missed. Cancelled‑flight family rebooking: across 24 attempts (8 models × 3), 6 of 67 to‑do lines and 33 of 103 amounts were written the way all four past notes write them. Told the rules, Opus 5.5 matched every one.

5. The kitchen saw too much. Multi‑shop bakery ordering app: 0 of 8 apps, one per model, kept customers' phone numbers and emails off the kitchen pages; in one, the kitchen page couldn't be reached. With a hint, a few added lines of advice, 7 of 8 did.

6. The household's formats missed. Stranded‑family rebooking: all 23 notes, of 24 attempts (8 models × 3), broke at least one of the four formatting conventions every past summary follows: 1 of 55 to‑do lines, 34 of 176 times and 27 of 88 bookings followed them.

Nine full solves in 395 attempts

Each dot is one attempt with only the request; the line marks the median. Larger dots are full solves.

Hover a dot for its model and reward.

Where each one keeps its requirements

Checks: a range where the count varies by attempt. Told the rules: in the waste report and the temperature register it also gives facts from the files, in the first a figure the grader checks. A dash: not tested.

What we haven't measured

• What counts as a requirement is the author's reading of the files, traced by audits: in the waste report all 35 requirements trace to the files, though a word limit and one placement rule aren't settled there; in the price report three graded points leave a second reasonable reading.
• Some app checks rest on a professional default no file shows, such as saving a lead with only a name and a phone.
• The cancelled‑flight rebooking's house‑style weights were set after seeing pilot results; with its five habits weighted equally, Opus 5.5 still leads and the middle of the table reorders.
• Shortcuts still pay in some: a figures‑only report scores 84% in the waste report and a bare table 97% in the price report; an empty registration meets 11 of 14 checks; posting nothing scores 50% in the tea‑factory chat, and an idle agent 18% at the spares stores.
• In the holiday‑rental chat, our attempts weren't isolated from the answer key.