AsapiThe AI Agent Trials
Asapi / Catalogue / Temperature compliance register from messy logger data
Environment datasheet · Small business

Temperature compliance register from messy logger data

ID house‑rules/task2‑1 · revision 21 · tested 21–22 September, 9 and 15 October 2026 · regraded 17 November 2026

Covering for a food‑safety coordinator who is off sick, the agent compiles a six‑shop bakery chain's monthly temperature register: every breach and every data fix for each fridge, freezer, display and hot cabinet, as three CSV files in the past registers' format.

Checks
103, in 23 equal groups, each paying only if all its checks pass
Tested
12 models, one attempt each
Best
40% (Sonnet 5.5)
Median
9%
Full solves
0 of 12
Median attempt
46 min, 122 tool calls
What it tests

A month of temperatures from six shops, turned into a register

Multimodal records

Scanned handwritten sheets and phone photos hold readings found nowhere else, beside logger files in mixed units and clocks.

Catching errors

Genuine unusual entries must be trusted and extra breaches never invented, while faulty sensors, clock errors and a fabricated sheet are caught.

Unstated requirements

Past registers show the firm's breach rules, such as a short dip back in range not ending an excursion; they override a consultant's different rules.

How it works

134 files, loggers on different clocks and paper with no text layer

The agent's world

A terminal in a fresh container with 134 files and no document, spreadsheet or scan tools, through the model's usual coding agent.

Data problems

Loggers in raw counts or Fahrenheit, on UTC, one clock fast; a stuck logger, a spike, a duplicated upload and decimal commas.

Limits

Three hours and a spend cap per attempt, with nobody to ask.

The reward

103 checks in 23 equal groups, all by script; a group pays only if all its checks pass.

Results

58% of attempts scored 6–40%; the rest left nothing the grader could read

Each dot is one attempt, and the line marks the median.

MedianFully solved
12 modelsOne attempt each9%0 of 12
ModelReward
Sonnet 5.5Anthropic40%
Opus 5.5Anthropic40%
Fable 5.1Anthropic23%
Sonnet 5Anthropic15%
GPT‑6 AstraOpenAI13%
GPT‑6.1 SolOpenAI13%
DeepSeek V4.1 FlashDeepSeek6%
GLM‑5.3‑FlashZhipu0%
GPT‑5.6 LunaOpenAI0%
GPT‑6 LunaOpenAI0%
Opus 5Anthropic0%
Qwen3.8‑27BAlibaba0%

Of the five that left nothing readable, two saved one workbook instead of three CSV files, one saved only pictures of the scans, and two stopped at a limit.

What models get wrong

They invent breaches, believe planted entries, and miss the past registers' format

Catching errors

  1. Every register the grader could read listed breaches that never happened, and six of those seven counted a planned deep clean, in whole or in part, as one.

  2. None of the seven readable registers flagged a compliant figure written over an alarm.

Unstated requirements

  1. The two best attempts ran the summary file over five weeks, not the month, which voids several groups at once.

Running it

What an attempt costs, and what the environment includes

Cost

The median attempt cost about $5.40 at list prices for the agent model.

What's in it

The 134 files, the request in all three versions, the grader and its check definitions, and per‑check results for every attempt.

Format

Packaged as a Harbor task on request; it runs in Docker today, and grading is a script alone.

Fresh variants

Other files, people and facts in the same design, made on request and never published.

The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows each model's score.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.