AsapiThe AI Agent Trials
Asapi / Catalogue / Stranded‑family rebooking with minors' exit papers and a dog
Environment datasheet · Personal assistant

Stranded‑family rebooking with minors' exit papers and a dog

ID trip‑rescue/task02 · revision 1 · results as of 29 October 2026

On the evening a family's flight home from abroad is cancelled, the agent, as the father's personal assistant, gets the mother and two children home by Wednesday, cheaply, settles the rental car and the dog, files refunds and claims, and writes a note and a money plan.

Checks
34, in five weighted parts
Tested
8 models, 3 attempts each
Best attempt
96% (Opus 5.5)
Median
61%
Full solves
0 of 24
Median attempt
9 min
What it tests

A family stranded abroad, with the children's exit papers about to expire

Clarifying with people

The facts that decide the route sit with one parent each, released only for the right question, such as the papers' end date.

Catching errors

The father believes the mother is on the rental contract; she isn't, and only the contract itself shows it.

When to act or escalate

Buying flights while a free rebooking exists fails a check, yet the second driver and the airline's statement must be secured that night.

Stateful tool use

Changes to flights, the car, the dog's travel and claims spread across a hub and nine sites, and must all fit together.

Unstated requirements

Four note habits appear only in the assistant's past trip summaries, such as giving times in 24‑hour form with the city.

How it works

Two mailboxes, nine sites and two parents who each hold part of the facts

The agent's world

A text browser into the family's hub and nine sites with delegated logins, through the model's usual coding agent, with no shell or web.

The hub and sites

Two mailboxes, 14 family documents and four past trip summaries, plus nine sites from the airline and a rental car to an insurer.

The parents

Code hands out each parent's fact cards only when asked, and a model words the replies; the clock stays frozen on Sunday evening.

The reward

34 checks: legal and on time 35, net cost 25, requests 15, the note 15, style 10; an illegal route home costs up to 55.

Results

Half the attempts scored 83–96%, and nearly all the others 35–40%

Each dot is one attempt, and the line marks the median.

MedianFully solved
8 models3 attempts each61%0 of 24
ModelMean of 3 attempts
Opus 5Anthropic92%
Opus 5.5Anthropic91%
GPT‑6 AstraOpenAI90%
GPT‑6.1 SolOpenAI72%
Fable 5.1Anthropic56%
GPT‑6 SolOpenAI36%
Sonnet 5.5Anthropic35%
GPT‑6 LunaOpenAI30%

With one paragraph stating every rule, house habit and whom to ask added to the request, both models tested scored 99% or more.

What models get wrong

They fly the children on expired papers, miss the house formats, and leave claims without an owner

Clarifying with people

  1. In 42% of attempts the children flew out after their exit papers ended; in half of those, the agent never got the date from the mother.

Unstated requirements

  1. All 23 notes written broke at least one of the four house formats every past summary follows: 1 of 55 to‑do lines, 34 of 176 times, 27 of 88 bookings and 97 of 161 amounts followed them.

  2. 75% of attempts left a claim's deadline or who files it out of the note, including 8 of the 13 that got the family home legally.

Running it

What an attempt takes, and what the environment includes

An attempt

The median attempt used about 9 million tokens, cached input included.

What's in it

The hub and its sites, the parents' service and fact cards, both versions, the full grader, the answer key and a reference solution.

Format

It runs today on a Mac in each agent's own sandbox, and is packaged as a Harbor task on request.

Fresh variants

Other files, people and facts in the same design, made on request and never published.

The files and grader aren't published; the job's page on The AI Agent Trials, our public benchmark, describes the job and each model's results in plain language.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.