Education · Task 23 of 34
The term registration
Fix a student's term registration on a university portal; the prompt only says other commitments exist, and every rule hides in the student's records.
The task
Registration week at Northgate University. A second-year computer science student is already signed up for four classes and asks the agent to sort out their Fall registration so that classes fit around the other commitments booked for the term, which are listed in the portal, without paying more than they have to. The agent works the student portal page by page: a course catalogue that lists nothing until searched, course pages, a campus map, the degree audit and the student's records. Nobody answers questions.
The prompt only asks for a registration that fits. The real test is reading the student's records the way a careful advisor would: turning a work contract, a training slot and a parking permit into times to avoid, keeping the scholarship, checking what each course requires, and noticing that the registration the student started with is already wrong.
What it tests
Working out what the job really needs
No page states a rule as a rule; a work shift, a training session and a permit each have to become a constraint on the timetable.
Weighing sources that mislead
One record states a limit and prices going over it in the same sentence, and an advisor warns that two courses are easy to confuse.
Checking what the site will not
The portal registers clashes, full sections and courses without their prerequisites, and only flags them.
Fixing what was handed over
The student's existing registration breaks several rules before the agent changes anything.
Challenges
1A limit written as a price
A permit states its limit and then what going over it costs, so the agent has to tie it to the student's wish to pay no more than needed. Real contracts are often worded this way.
2A broken starting point
The student's own registration already breaks several rules, and the site points out only one of them.
3Prerequisites the site ignores
Course pages list prerequisites, the portal registers the course anyway, and only the audit says what the student has passed.
4Two courses that are easy to mix up
The advisor names the course to take by what it covers and warns that its number and title get confused every year.
5Rules that are not where you look
The scholarship's terms and the advisor's note sit in separate records from the commitments the prompt mentions.
6Time to cross the campus
Training happens at a named building, so the classes either side of it have to leave long enough to walk there and back.
Results
| Model | Standard | Time (standard) | Cost (standard) |
|---|---|---|---|
| Fable 5.1 | 79% | 4 min | $9.46 |
| Opus 5 | 79% | 4 min | $2.91 |
| Opus 5.5 | 79% | 3 min | $2.95 |
| Sonnet 5 | 57% | 2 min | $0.34 |
One run per model and setup. 100% = everything right (or, for a plan scored in money, the best plan); 0% = the obvious answer or worse.
Setups: Standard, the job as asked.
What models got right and wrong
- No model met every rule. Fable 5.1, Opus 5 and Opus 5.5 tied at 78.6%, each losing the same three checks; Sonnet 5 scored 57.1%.
- The three leaders handled the student's time well: they dropped the class that starts before the car park opens, kept the work days and the midday training slot clear, stayed inside the parking permit's hours and left enough time for every walk between classes.
- The commonest failure was never reading the records about the degree, which hit Fable 5.1, Opus 5, Opus 5.5 and Sonnet 5. None opened the advisor's note, so every run missed the systems course the advisor asked for and closed too few degree requirements.
- Every run also kept a course whose prerequisite the student has not passed, without ever opening that course's page; three of them registered the prerequisite for the same term.
- Fable 5.1 did hold the systems course at one point, then swapped it away to fix two tight walks that were caused by the two classes it had chosen to keep.
- Sonnet 5 treated the job as two missing credits: it added one small afternoon class and changed nothing else, keeping the early class, a walk the site flags as too tight and a day of nearly eight hours.
- There is no hinted or told-everything setup for this task: every score is on the job as asked.
Test environment
- Kind
- Application: the student enrollment portal of Northgate University, a made-up university, in Fall 2026 registration week, used on behalf of one second-year computer science student. A local web server that records the registration
- Agent's interface
- No container and no terminal. The portal harness (
portal_agent.py) loads the site in headless Chromium through Playwright and gives the model each page as numbered text; the model replies with one action. Every model runs in this same harness, through the Claude Code app (claude -p, signed in, with Claude Code's own tools turned off) - Documents
- No files. The site serves 87 HTML pages at the start of a run, seen only as text: the catalog's search page, 51 course pages (13 departments, 108 sections, 13 of them full), the registration cart, My Schedule, the degree audit, the student record, the term commitments, the advising note, the campus map and 28 map place pages. Search results are built when asked for
- Applications
- The portal only: a catalog search (keyword, department, meeting day, time of day, availability), course pages with Add and Remove buttons, a cart with "Submit registration", and views of the schedule, the audit, the records and the map. The site flags clashes, full sections and tight walks but blocks nothing. No shell, file or office tools
- Web
- None. The agent cannot leave the portal: going to any other address is refused, and it has no web search or fetch
- People
- None. Nobody answers questions; the advisor's note is a page on the site
- Time
- A fixed in-world date: every page's header says "Today: Monday 24 August 2026", and the system prompt says to go by it. No clock runs; nothing on the site changes except through the agent's actions
- Actions
- One per reply: click, type, select, go to a path on the site, go back, stop. Adding a section and submitting the cart, or removing a registered section, change the registration; nothing else can be changed, and nothing is sent to anyone
- Deliverable
- The registration left in the portal when the agent stops, and a closing message (the text of its stop action). Only the registration is graded