Education · Task 24 of 34
Graduating soonest
Fix a student's term registration so they graduate as soon as possible; the course that helps most today is not the one that gets them out soonest.
The task
Registration week at Northgate University. A second-year computer science student, already signed up for four classes, asks the agent to sort out their Fall registration around the commitments listed in the portal, and adds that they want to graduate as soon as possible without paying more than they have to. The agent works the same student portal page by page: catalogue, course pages, campus map, degree audit and the student's records. Many sections are full. Nobody answers questions.
The prompt only asks for a registration that fits and a quick graduation. The real test is planning past this term: weighing what a course closes now against what it opens later, checking every prerequisite against what the student has actually passed, and still meeting every rule in the student's records.
What it tests
Judging a decision over the long run
The choice that closes a requirement this term competes with one that shortens every term after it.
Working out what the job really needs
The commitments, the scholarship and the degree plan each have to be found and turned into rules.
Weighing sources that mislead
One record states a limit and prices going over it, and the audit describes requirements without naming the courses that meet them.
Checking what the site will not
The portal registers courses without their prerequisites, full sections and clashes.
Challenges
1Today's progress against the fastest route
The option that ticks a box this term is not always the one that gets the student out soonest, and the student's existing registration already holds the tempting one.
2A course the site lets you take too early
An attractive upper-level course closes more than one requirement, and the portal registers it without its prerequisite.
3An audit that describes, not names
The degree audit says what kind of course is needed, so the agent has to work out which courses count.
4A limit written as a price
A permit states its limit and then what going over it costs.
5Full sections
Some classes look available but have no seats, and a waitlist does not count.
6Time to cross the campus
Training happens at a named building, so the classes either side of it have to leave long enough to walk there and back.
Results
| Model | Standard | Time (standard) | Cost (standard) |
|---|---|---|---|
| Opus 5 | 79% | 12 min | $9.36 |
One run per model and setup. 100% = everything right (or, for a plan scored in money, the best plan); 0% = the obvious answer or worse.
Setups: Standard, the job as asked.
What models got right and wrong
- Opus 5 is the only model run on this task so far, and scored 78.6%.
- It met every rule about the student's time: work days, weekends, the car park's hours, the midday training slot, clashes, the walks between classes and the parking permit's six-hour day.
- It closed three outstanding degree requirements, one more than the task asks for, and read prerequisites carefully enough to rule out two upper-level courses that needed courses the student has not passed.
- It reached the choice the task turns on and decided against it: it worked out that the course unblocking the upper-level work shares its only open slot with the course the student already held, kept the one it had, and told the student to take the other next term.
- Dropping the early-morning class left it short of the credit minimum, and the only course it found that filled the slot needed a prerequisite the student has not passed and sat too far from the pool for the walk back from training.
- There is no hinted or told-everything setup for this task: the score is on the job as asked.
Test environment
- Kind
- Application: the student enrollment portal of Northgate University, a made-up university, in Fall 2026 registration week, used on behalf of one second-year computer science student who wants to graduate as soon as possible. A local web server that records the registration
- Agent's interface
- No container and no terminal. The portal harness (
portal_agent.py) loads the site in headless Chromium through Playwright and gives the model each page as numbered text; the model replies with one action. Every model runs in this same harness, through the Claude Code app (claude -p, signed in, with Claude Code's own tools turned off) - Documents
- No files. The site serves 87 HTML pages at the start of a run, seen only as text: the catalog's search page, 51 course pages (13 departments, 108 sections, 22 of them full), the registration cart, My Schedule, the degree audit, the student record, the term commitments, the advising note, the campus map and 28 map place pages. Search results are built when asked for
- Applications
- The portal only: a catalog search (keyword, department, meeting day, time of day, availability), course pages with Add and Remove buttons, a cart with "Submit registration", and views of the schedule, the audit, the records and the map. The site flags clashes, full sections and tight walks but blocks nothing. No shell, file or office tools
- Web
- None. The agent cannot leave the portal: going to any other address is refused, and it has no web search or fetch
- People
- None. Nobody answers questions; the advisor's note is a page on the site
- Time
- A fixed in-world date: every page's header says "Today: Monday 24 August 2026", and the system prompt says to go by it. No clock runs; nothing on the site changes except through the agent's actions
- Actions
- One per reply: click, type, select, go to a path on the site, go back, stop. Adding a section and submitting the cart, or removing a registered section, change the registration; nothing else can be changed, and nothing is sent to anyone
- Deliverable
- The registration left in the portal when the agent stops, and a closing message (the text of its stop action). Only the registration is graded