Course registration optimised for earliest graduation

The first registration's rules, bar the advisor's course, plus a choice for later terms

Role

Acting for the same second‑year student, on the same registration portal.

Request

Sort out the autumn registration so classes fit around the commitments listed in the portal, graduating as soon as possible and paying no more than needed.

Deliverable

The registration the portal holds when the agent stops. The closing message is recorded but not graded.

What the reward depends on

13 rules on commitments, the scholarship, a workable timetable, degree progress and prerequisites, as in the companion environment, and one optimality check: which of three courses sharing a time slot gets the student to graduation soonest.

Where the requirements are

In the student's records on the portal, as before. The degree audit now describes core courses without naming them, and the advisor's note now suggests, rather than asks for, the course it describes.

The same 87‑page portal, now with 22 full sections

Interface

A Python harness loads a live portal in headless Chromium and gives the model each page as numbered lines of text; the model answers with one action a turn. No container, terminal, files or web: any address off the portal is refused.

Portal

87 pages at the start: the catalogue search, 51 course pages (13 departments, 108 sections, 22 full), the cart, the schedule, the degree audit, the student's records, an advisor's note and a campus map with 28 places.

Actions

Click, type, select, go to a path on the site, go back, or stop with a closing message. Adding a section only fills a cart until the registration is submitted; removing a registered section drops it at once.

What the site allows

It blocks nothing: it registers clashes, full sections (as a waitlist place) and courses without their prerequisites. It flags clashes, full sections and walks too long for the gap, but not a missing prerequisite.

Model calls

Each reply is one fresh Claude Code call with Claude Code's own tools off. The harness passes the conversation so far, with pages older than 40 turns replaced by a placeholder.

Stops

Four replies in a row with no valid action, or one action repeated five times, end an attempt; otherwise it ends when the agent stops or reaches 250 replies.

Time and people

A fixed in‑world date; no clock runs and nobody answers. The registration changes only through the agent's actions.

13 rule checks and one optimality check, all by script

Structure

14 yes/no checks, one point each: 13 rules and one optimality check. Reward = checks met / 14; a pass is all 14.

Optimality

It passes only for the course the answer key takes as the route to graduating soonest; keeping the course already held breaks no rule.

Who decides

Scripts alone, on the portal's own record of the registered sections when the agent stops; sections left in the cart don't count. No LLM judge, no penalties, and the closing message isn't read.

Rebuilt and replayed

An offline grader rebuilds each final registration in a fresh portal and reaches the same verdict. Its self‑test passes the answer key, fails the starting registration and regrades every attempt to its recorded checks.

Reference and floor

A scripted reference scores 100%, and 9 registrations the catalogue allows meet all 14. The starting registration meets 7 (50%); an empty one meets 11 (79%).

One attempt: two rules broken by its last course, and the optimality point missed

Only Opus 5 was tested, once, scoring 79%. It read the advisor's note and closed as many requirements as it asks for. Its last‑added course broke two rules, a missing prerequisite and the walk from training, and it kept the course already held over the one the optimality check rewards.

The dot is the one attempt.

One attempt by one Anthropic model: the harness reached models through Claude Code. Two earlier attempts were set aside unscored, one made before a record was corrected and one cut short by a provider's session limit.

There is no version with a hint or with the rules written in.

What the model got wrong

One attempt by one model, with only the request, so the first two are one model's mistakes, not rates; the last adds the companion environment's four attempts, pooled and named as such.

1. Prerequisite unchecked. To reach the credit floor it added a course whose prerequisite the student hadn't passed, and the portal registered it. It had checked prerequisites for two other courses, but not this one.

2. The walk from training. The same course started too soon after training for the walk from the pool, though the map lists every walking time.

3. The same miss next door. In the companion environment, on the same portal and student record, all 4 attempts by four models left the student in a course they weren't ready for: 5 of 5 attempts across the two, pooled.

What an attempt costs

Length

The one attempt took 12 minutes and 43 actions, and about 900,000 tokens of input and output; the harness records no cache figures.

Cost

It cost $9.36 at list prices. That's the agent model only, and nothing else in the environment calls a model.

What you license

With a licence

Revision 2: the portal and its pages, the agent harness, the grader and its self‑test, the answer key with a scripted reference and an optimal registration, and the grader's report on the one attempt, as a baseline result.

Format

Packaged as a Harbor task on request. Today the portal and grader are standard‑library Python, and the harness drives the site with Playwright and Chromium, reaching models through the Claude Code CLI.

Revisions

Revision 2 follows an audit on 9 October: the records say how the permit's hours count, where training is, that it ends earlier and that shift days are barred; the training check counts the walk to the pool and back; the request asks to pay no more than needed.

Fresh variants

Other data, people and facts in the same design, on request. They're never published.

Public or not

Its job page on The AI Agent Trials, our public benchmark, describes the job and the one model's result in plain language, so hold out fresh variants for evaluation. The files, grader and answer key aren't published.

What we haven't measured, and what's uneven

• An empty registration meets 11 of 14 checks (79%), as much as the one attempt, since 11 checks only look for broken rules. Leaving the starting registration as it is meets 7.
• The optimality check is disputed: an audit on 9 October found nothing on the portal that shows the course it rewards gets the student out sooner, and the advisor's note points the other way. The owner kept it, as the student's stated wish decides.
• One attempt by one model, so these results say nothing about how attempts vary or how other models do.
• The credit check counts a waitlisted class's credits, though the scholarship counts them only once a seat is confirmed. The one attempt held no waitlisted class.
• No red‑team attempts yet.