Asapi
For AI labs / Catalogue / Club race‑car design and parts procurement on a web app with hidden supplier terms
Environment datasheet · Race‑car engineering

Club race‑car design and parts procurement on a web app with hidden supplier terms

ID swap‑garage/task01 · revision 1 · results as of 17 November 2026

On a club racing team's garage website, the agent picks one of 19 chassis, builds and simulates the fastest car the class allows, and freezes the design. Then it buys every part before race day and within budget, from six suppliers who give stock, delivery and price only when asked.

Checks
4, pass‑fail gates: a legal car, no part combination that has broken down, a design frozen first, and every part ordered in time, within budget and to the car's specification
Tested
5 models, one attempt each
Best
2.0% slower than the fastest achievable car (GPT‑6 Astra)
Median
4.6% slower
Full solves
0 of 5
Median attempt
19 min, 575 tool calls
What it tests

Design the fastest car the rules allow, then buy it from suppliers who quote only when asked

Engineering design

The reward compares the car's handicapped lap with the fastest achievable car's, the fastest buyable, breakdown‑free build among 42.7 million legal builds of 19 chassis.

Unstated requirements

Six fast pairs of parts have broken down in past races; only the series' past results show it, and a car using one scores 0.

Stateful tool use

The design must be frozen before suppliers answer, every installed part needs a matching on‑time order, and a redesign can void one.

Clarifying with people

Six scripted suppliers give stock, delivery and price only when asked, the only way to find the part that can't arrive in time.

How it works

A garage website, six supplier portals, four seasons of results and one submission

The agent's world

A headless browser as its only tool, on a sealed network with no shell, files or web, for 90 minutes.

The garage

19 chassis with published handicaps, a lap simulator to run at will, and one design frozen before suppliers answer; redesigns may follow.

The suppliers

Six scripted workshops answer stock, price and delivery questions, with an $11,400 budget and race day on day 7.

The reward

A car that fails any of four gates earns a reward of 0; otherwise, the fastest achievable car's handicapped lap divided by its own.

Results

Every car passed all four gates, and the closest lapped 2.0% slower than the fastest achievable car

Each dot is one attempt, placed by how much slower its car lapped than the fastest achievable car, and the line marks the median.

MedianFully solved
5 modelsOne attempt each4.6% slower0 of 5
ModelSlower than the fastest achievable car
GPT‑6 AstraOpenAI2.0%
Opus 5.5Anthropic4.4%
GPT‑6 SolOpenAI4.6%
Opus 5Anthropic4.7%
GPT‑6 LunaOpenAI6.8%

As rewards, these attempts earned 94% to 98%: the fastest achievable car's handicapped lap divided by each car's own. Only the fastest achievable car itself would be a full solve; a stock car of one chassis, bought in full, earns about 78%, 29% slower.

What models get wrong

They settle for a slower car than they could have bought, and four end on the same chassis

Engineering design

  1. The submitted cars lapped 2.0% to 6.8% slower than the fastest achievable car; 673 to 79,651 buyable builds were faster.

  2. Four of five submitted the same chassis and lapped 4.4% to 6.8% slower, and two never chose any other; GPT‑6 Astra switched chassis after its first design couldn't be bought, and lapped 2.0% slower.

  3. When its tyre couldn't arrive by race day, Opus 5.5 changed tyres and tested other parts but kept the chassis, though the fastest achievable car was on another.

Running it

What an attempt takes, and what the environment includes

An attempt

Attempts took 11 to 62 minutes, with 16 to 47 simulations and 437 to 836 tool calls.

What's in it

The garage, supplier and results site, the lap simulator, and the grading server with its exhaustive table of legal builds.

Format

Three Docker images, the site's server, an egress proxy and the agent's browser, on one sealed network; other packaging on request.

Fresh variants

Other seeded episodes in the same design, made on request and never published.

The files and grader aren't published; the job's row on The AI Agent Trials, our public benchmark, sums up the job and shows how much slower each model's car was.

Try one of our three sample environments

Three environments are packaged as samples you can download, with their graders and reference solutions. Ask for them with your work email, or book a call about the rest of the catalogue.