AsapiThe AI Agent Trials
Asapi / Catalogue / Course registration optimised for earliest graduation
Environment datasheet · University portal

Course registration optimised for earliest graduation

ID northgate/task1‑2 · revision 2 · results as of 9 November 2026

The same portal, student and commitments as the course‑registration environment, with one more wish: the student wants to graduate as soon as possible, and 22 sections are full. The agent has to weigh what each course opens up for later terms.

Checks
14, 13 rules and 1 optimality check
Tested
15 models, one attempt each
Best
93% (GPT‑5.6 Sol and GPT‑6 Astra)
Median
79%
Full solves
0 of 15
Median attempt
10 min, 45 tool calls
What it tests

The same timetable, now planned for graduating as soon as possible

Stateful tool use

The portal registers whatever it's given, so the final timetable has to keep every rule and graduate the student soonest.

Planning ahead

One check rewards the same‑slot course that leads to graduating soonest, though keeping the course already held breaks no rule.

Catching errors

The starting registration holds problems, such as a course without its prerequisite that also meets on the student's work days.

Unstated requirements

Commitments, parking limits, scholarship terms and degree rules must be found across the portal's pages; the request names none.

How it works

A live portal read as text, one page and one action at a time

The agent's world

A harness shows each portal page as numbered lines of text; the model answers with one action: click, type, select, go, back or stop.

The portal

87 pages, including 51 course pages with 108 sections, 22 of them full; it blocks nothing, registering clashes and missing prerequisites.

Limits

Up to 250 replies, with a fixed in‑world date and nobody to answer; loops of repeated actions end an attempt.

The reward

13 rule checks and one optimality check by script, on the registration the portal holds; an empty one meets 11.

Results

The best two scored 93%, each missing only the optimality check

Each dot is one attempt, and the line marks the median.

MedianFully solved
15 modelsOne attempt each79%0 of 15
ModelReward
GPT‑5.6 SolOpenAI93%
GPT‑6 AstraOpenAI93%
GPT‑6 SolOpenAI86%
GPT‑6.1 SolOpenAI86%
GPT‑6.1 TerraOpenAI86%
Opus 5.5Anthropic86%
Fable 5.1Anthropic79%
Opus 5Anthropic79%
Sonnet 5.5Anthropic71%
GPT‑6 LunaOpenAI64%
GPT‑5.6 LunaOpenAI57%
GPT‑5.6 TerraOpenAI57%
Haiku 4.5Anthropic57%
Haiku 5.5Anthropic50%
Sonnet 5Anthropic50%
What models get wrong

They keep the course already held, trust what the portal accepts, and read commitments loosely

Planning ahead

  1. 93% kept the course already held in the contested slot, which breaks no rule, rather than the one leading to graduation soonest.

Catching errors

  1. 67% held a course whose prerequisite the student hadn't passed, which the portal registered without objection.

Unstated requirements

  1. 40% kept the starting class that begins before the car park opens, and 47% had a day on campus longer than the permit covers.

Stateful tool use

  1. 33% left a transfer too short to walk.

Running it

What an attempt takes, and what the environment includes

An attempt

The median attempt used about 1.4 million tokens of input and output.

What's in it

The portal and its pages, the agent harness, the grader and its self‑test, and an answer key with an optimal registration.

Format

Packaged as a Harbor task on request; the portal and grader are standard‑library Python, and the harness drives the site in Chromium.

Fresh variants

Other files, people and facts in the same design, made on request and never published.

The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.