AsapiThe AI Agent Trials
Asapi / Catalogue / Household savings plan from records and family
Environment datasheet · Household

Household savings plan from records and family

ID house‑rules/task13 · revision 4 · tested 13–14 October 2026 · regraded 17 November 2026

From a busy father's laptop and phone, the agent writes one plan: a six‑month emergency fund first, then a set sum saved on top, as fast as he can. He answers nothing; his wife, family, builder and HR hold figures his files leave out or get wrong.

Checks
51, 46 on decisions, 5 on results
Tested
8 models, one attempt each
Best
85% (Opus 5.5)
Median
50%
Full solves
0 of 8
Median attempt
8 min, 53 tool calls
What it tests

An emergency fund first, then savings, from a father too busy to answer

Clarifying with people

Six checks need a figure only a person gives when asked, from HR to the clinic, and the father himself never answers.

Multimodal records

Chats, social posts and side earnings exist only as phone screenshots, cited by 24 of the 51 checks; statements need a worked‑out password.

Planning ahead

A cash‑flow model built from the plan's own figures checks its two target months and that the money never runs below zero.

How it works

Files, mail, a phone, locked statements and nine people to ask

The agent's world

A laptop command with files, mail, a phone, bank statements and offline web pages, through the model's usual coding agent; nothing can be paid.

Hidden pages

Nine signed‑in account pages open only from his bookmarks, and one card's account only from its bank's own pages.

The people

Nine people voiced by an AI model from 80 fact cards; some need a second ask, and HR answers the next working day.

The reward

46 decision checks and 5 on results, three via a cash‑flow model; six need the right person asked, and the wrong take‑home pay scores 0.

Results

The top two scored 80–85%, and the rest 0–51%

Each dot is one attempt, and the line marks the median.

MedianFully solved
8 modelsOne attempt each50%0 of 8
ModelReward
Opus 5.5Anthropic85%
Fable 5.1Anthropic80%
GPT‑6.1 SolOpenAI51%
Sonnet 5.5Anthropic51%
GPT‑6 AstraOpenAI49%
Opus 5Anthropic48%
GPT‑6 LunaOpenAI34%
Sonnet 5Anthropic0%

With one added sentence telling it to ask the people involved where a figure is missing, all eight messaged people, and seven scored higher.

What models get wrong

They ask no one, search or ask but never both, and build on maybes

Clarifying with people

  1. 75% of attempts sent no message or call, though only the father was unavailable.

Multimodal records

  1. Two bills sit only on bookmarked account pages: the attempts that found both asked no one, and those that asked people never opened them.

Planning ahead

  1. No plan budgeted for a parent's further medical tests, which the clinic's billing desk prices when asked.

  2. 63% built the recommended plan on uncertain weekend side income, with no version that works without it.

Running it

What an attempt costs, and what the environment includes

Cost

The median attempt cost about $2.30 at list prices for the agent model; the simulated people's and judges' calls are extra.

What's in it

Revision 4 in all three versions: the world service with its fact cards, the files, screens and pages, the full grader, and per‑check results.

Format

Packaged as a Harbor task on request; it runs in Docker today, with the people and readers reached through model CLIs.

Fresh variants

Other files, people and facts in the same design, made on request and never published.

The files and grader aren't published; the job's page on The AI Agent Trials, our public benchmark, describes the job and each model's results in plain language.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.