Asapi
The AI Agent Trials / The jobs
The jobs

All 56 jobs

Every job we've given the AIs, by benchmark. For each, how many AIs we tested on it, and which got everything right or came closest. A score out of 100 says how much of the job an AI got right; where it had several tries, it's the average. How scores work.

Your AI personal assistant At home · 10 jobs

  • Organise the family's New Year dinnerDaniel Lim, in Malaysia, wants his family's Chinese New Year reunion dinner, about 40 people from seven households, moved from home to a restaurant, so all he has to do is turn up.9 AIs tested · closest: GPT‑6.1 Sol, 86 out of 100
  • Rescue a cancelled tripThe Calders' flight to a family wedding near Lisbon is cancelled, and the AI, at their travel firm, has to get all four there by Friday, spending up to $1,500 without asking.8 AIs tested · closest: Opus 5.5, 81 out of 100
  • Check a household's monthly billsAt a bill‑checking firm, the AI reviews Dana Ferreira's bills for the first time since she moved house, disputes the wrong charges and writes the letters.8 AIs tested · closest: Opus 5.5, 91 out of 100
  • Take on a new client's billsThe same firm takes on Celia Marsh. Three of her charges need her answer, like whether she'd sent some boots back, and the AI can message her, though it isn't told it can.8 AIs tested · closest: GPT‑6 Astra, 82 out of 100
  • Sort out a couple's medical billsFor the same firm, the AI goes through a year of Dennis and Joanna Vellacott's medical bills across two health plans, writing to the plans, the doctors and the office at Dennis's work that runs his plan.8 AIs tested · closest: Fable 5.1, 86 out of 100
  • Decide on solar panelsDaniel wants the best solar plan for his home, and a fair deal with his brother Kenneth, who covers the power bill in place of rent.10 AIs tested · closest: GPT‑6.1 Sol, 78 out of 100
  • Plan an emergency fundDaniel wants an emergency fund worth six months of his family's costs, and then RM 20,000 on top. He's too busy for questions, but his wife, his family and HR can answer them.8 AIs tested · closest: Opus 5.5, 89 out of 100
  • Sort out next term's classesA student, Jordan Avery, needs next term's classes to work around library shifts, swimming and a time‑limited parking permit, with enough credits to keep a scholarship.4 AIs tested · closest: Fable 5.1, Opus 5 and Opus 5.5, 79 out of 100
  • Sort out classes and graduate soonerJordan also wants the quickest route to a degree, and three courses compete for one slot in the week.1 AI tested · closest: Opus 5, 79 out of 100
  • Bring a stranded family homeMarina's flight home from Brazil to Boston with the two children and their dog is cancelled, and the AI, as her husband Dan Ashgrove's personal assistant, has to get the family home by Wednesday night, the day his parents arrive for Thanksgiving.8 AIs tested · closest: Opus 5, 92 out of 100

Your AI office assistant At work · 12 jobs

  • Sign off a cake box in the owner's nameThe AI picks the cake box the owner would choose and fills in the printer's approval forms in her name, before a 5 pm cut‑off.13 AIs tested · closest: GPT‑6 Astra, 94 out of 100
  • A shop's broken ice cream freezerA shop manager says his ice cream freezer has failed, and asks the AI which new one to buy before Easter.13 AIs tested · GPT‑6 Astra got everything right
  • Write the monthly waste reportWith the analyst on leave, the AI writes the bakery chain's monthly waste report for the owner, the way she has since asked for it.13 AIs tested · closest: GPT‑6 Astra, 94 out of 100
  • Write a brewery's monthly price reportWith the purchasing manager away, the AI writes the brewery's monthly report on ingredient and packaging costs, which goes on to its bank.12 AIs tested · closest: GPT‑6 Astra and Fable 5.1, 90 out of 100
  • Check a contractor's bill before it's paidThe AI approves the refrigeration contractor's three‑month bill for tomorrow's payment run, checking every line against the shops' records.13 AIs tested · closest: Opus 5.5, 90 out of 100
  • Finish an insurance claim for a broken freezerA shop freezer has failed, and the AI finishes the insurance claim and its documents, which finance will send exactly as they're left.14 AIs tested · closest: DeepSeek V4.1 Flash and Fable 5.1, 89 out of 100
  • Build the monthly wholesale figuresThe AI builds the month's sales figures for the wholesale customers, which finance, planning and stores all rely on.13 AIs tested · closest: GPT‑6.1 Sol, 91 out of 100
  • Land a hotel's Christmas orderWith the owner away, the AI decides on a hotel group's big Christmas order and organises it through her laptop and her team.8 AIs tested · closest: Opus 5.5, 88 out of 100
  • The freezer fails againFive months on, the same freezer can't hold its temperature, and the AI decides whether to patch, mend or replace it. The plan is judged on the next three years.12 AIs tested · closest: GPT‑6 Astra, 99 out of 100
  • Keep the fridge and freezer temperature logThe AI turns a month of logger readings and handwritten sheets from six shops into the food‑safety register.12 AIs tested · closest: Sonnet 5.5, 76 out of 100
  • Put together a bakery's fire claimA fire has wrecked the kitchen behind one of a small bakery chain's shops, and the AI fills in the insurer's loss form for what was lost in it, for the owner to check and send on Monday.7 AIs tested · GPT‑6 Astra got everything right
  • Furnish a room from six photosDana Okafor wants the living‑dining room of her new apartment exactly like the model apartment's, which she has only in six photos, and the AI, at a furnishing service, orders every piece within $9,000 and draws the floor plan for the delivery crews.8 AIs tested · closest: GPT‑6 Astra, 74 out of 100

Your AI app builder Software, no coding · 13 jobs

  • An app to sort out the household billsAfter a move, a woman wants one place for all her bills, showing the wrong charges with claim emails ready to send.8 AIs tested · closest: Opus 5, 61 out of 100
  • An app for a home‑loan brokerA broker starting out on the side asks for a web app to run his clients and their documents.8 AIs tested · closest: Opus 5.5, 90 out of 100
  • The broker's app, with a WhatsApp assistantThe same broker now also wants an assistant that answers his clients on WhatsApp while he works.8 AIs tested · closest: Sonnet 5.5, 68 out of 100
  • Online ordering for a bakeryA bakery owner with several shops wants customers to order cakes online, and her shops and kitchen to manage the orders.8 AIs tested · closest: Fable 5.1, 75 out of 100
  • An app for a piano teacher's studioA piano teacher wants lessons, pupils' work, recordings, invoices and payments in one app.7 AIs tested · closest: Sonnet 5.5, 81 out of 100
  • An app to run a laundry shopA laundry owner wants customers to book collections, staff to track every batch, and everyone to see progress.7 AIs tested · closest: Opus 5.5, 77 out of 100
  • A piano practice app for a familyA family of three pianists wants an app that guides a beginner and plays along with the others.7 AIs tested · closest: GPT‑6 Astra, 58 out of 100
  • An app for a gym and its membersA gym owner who also coaches 22 people wants one app for members, trainers and the front desk, built from old records and chats, just as the bank upstairs signs up its 900 or so staff.7 AIs tested · closest: Opus 5.5, 86 out of 100
  • A train‑ticket catcher for a commuterFaiz works in Singapore and goes home to Segamat every weekend, and wants an app that buys sold‑out weekend train seats when they come back, without the booking site blocking him for checking too often.8 AIs tested · Opus 5.5 got everything right
  • A smart home run from an iPadThe owner of a small terrace house in Selangor wants to run it from an iPad, and from a phone when out, without one brand's hub or cloud, and leaves the AI to choose, set up and pair everything.7 AIs tested · closest: Opus 5.5, 92 out of 100
  • A fault finder for cooling unitsA data centre's facilities lead wants a tablet app that walks a new electrician on the night shift from a broken cooling unit to its cause, the repair and a filled‑in report, using only old drawings, parts lists, a manual and the maintenance log.7 AIs tested · closest: Opus 5.5, 87 out of 100
  • Assembly manuals for flat‑pack furnitureSomeone who has just moved has ten pieces of flat‑pack furniture and no instructions, and wants one app with a step‑by‑step manual and a 3D model for each, worked out from 177 photos.7 AIs tested · closest: Opus 5.5, 63 out of 100
  • A journey planner for a city's transportThe AI is the engineer of a city's journey‑planning app: it builds the route planner and a program on riders' phones, then improves both over four weeks from what riders do and say where the timetable is wrong.5 AIs tested · closest: GPT‑6 Astra, 98 out of 100

Your AI customer service agent Customer service · 6 jobs

  • A customer who only wants a return labelA customer of a home‑goods store asks only for a new return label, and says everything else is fine.8 AIs tested · closest: Fable 5.1, 98 out of 100
  • Diagnose 80 customers at a repair deskThe AI finds each customer's appliance fault by asking a few questions, then books the right fix and prices it, 80 times over.5 AIs tested · GPT‑6 Astra, Opus 5.5, GPT‑6.1 Sol got everything right
  • Answer an overdue complaintA home‑goods store customer's complaint has gone unanswered since the weekend, and the AI has to reply.15 AIs tested · closest: Opus 5.5, 98 out of 100
  • Find a customer's missing paymentsA customer says four payments never reached her card, and the AI has to trace each one.8 AIs tested · closest: Opus 5, 93 out of 100
  • Find holiday rentals in a chatFive customers each send five messages looking for a place to stay, and the AI has to keep track of what each one needs.6 AIs tested · closest: Opus 5.5, 68 out of 100
  • List every rental that fitsFor each of 20 requests, the AI returns every holiday rental that meets it, and none that doesn't.6 AIs tested · closest: GPT‑6 Astra, Fable 5.1, Opus 5.5 and GPT‑6.1 Sol, 80 out of 100

Your AI operations manager Long jobs · 8 jobs

  • Run a delivery fleet for a monthThe AI plans a delivery company's trucks and gives every customer a delivery window, day after day for a simulated month.2 AIs tested · Opus 5.5 got everything right
  • Run a tea factory's cash for 13 weeksEach Monday, the AI decides what to sell to whom and when to ask the owner for money, without running out of cash.5 AIs tested · Opus 5.5 got everything right
  • Run the spare‑parts stores at two sitesThe AI keeps two sites' spare‑parts stores stocked, so technicians find what each repair needs.6 AIs tested · Opus 5.5 got everything right
  • Buy everything for a new packing lineThe AI buys every part the company has to supply for a new packing line, in time for each stage of the installation.6 AIs tested · GPT‑6 Astra, Fable 5.1, Opus 5.5, GPT‑6.1 Sol got everything right
  • Buy spare parts as suppliers withdraw themThe AI keeps spare parts on the shelf at two sites as suppliers stop making six of them.5 AIs tested · GPT‑6 Astra, GPT‑6.1 Sol got everything right
  • Plan a coffee roaster's materialsThe AI buys coffee and packaging in time for a roaster's weekly plan, through a port closure and the Christmas rush.5 AIs tested · closest: GPT‑6.1 Sol, 97 out of 100
  • Help a tea factory's crew in its group chatIn the crew's group chat, the AI speaks up when a machine setting is wrong for the day's tea leaf, and stays quiet when the crew has it right.6 AIs tested · closest: Opus 5, 67 out of 100
  • Run a coffee roaster's order desk at ChristmasThe AI runs a coffee roaster's web‑shop order desk through the Christmas peak, from the warehouse paperwork to every customer's case, waking each working day with only the notes it left itself.5 AIs tested · closest: Opus 5.5, 94 out of 100

Your AI hospital assistant Health care · 3 jobs

  • Coordinate a hospital ward for three daysFor three days and nights, the AI makes sure the doctors' orders on a ward get done.6 AIs tested · Opus 5.5 got everything right
  • Build a radiation‑dose records programThe AI builds the program that keeps each patient's radiation‑dose record, exactly as a 2,448‑line manual describes.12 AIs tested · GPT‑6 Astra, DeepSeek V4.1 Flash, Fable 5.1, Opus 5, Opus 5.5, Qwen3.8‑27B, GPT‑5.6 Sol, GPT‑6.1 Sol got everything right
  • Build an emergency department's patient summaryThe AI builds the program that gathers everything known about a patient arriving at an emergency department, as a 4,555‑line manual describes.6 AIs tested · Opus 5.5, Sonnet 5.5 got everything right

Your AI facilities engineer On site · 2 jobs

  • Handle a server room's cooling failureOn shift as a police station's duty engineer, the AI learns that a contractor has cut the cable that controls both server‑room cooling units, and has to keep the servers running until the units restart, then report what happened.7 AIs tested · closest: Fable 5.1, 77 out of 100
  • Find why a server hall's temperature swingsSince a contractor replaced the two cooling units in a police station's server hall, its temperature has swung all day, and the AI, as the engineer on site, has to find the cause with tests it can undo and report it.7 AIs tested · closest: Fable 5.1, 70 out of 100

Your AI engineering assistant Engineering drawings · 2 jobs

  • Digitise a plant's scanned drawingsHired at a fixed price, the AI turns an industrial plant's faded, hand‑marked scanned drawings into lists of every instrument, pipe and piece of equipment, and each question it asks the plant's staff comes out of that price.5 AIs tested · Sonnet 5.5 got everything right
  • Bring a plant's drawings up to dateSix months of paperwork from nine teams says what changed on an industrial plant, and the AI checks each claim against what was really done and draws the true changes onto the plant's master drawings.6 AIs tested · Fable 5.1 got everything right