A discount code can look like a win until it fails at checkout. AI business agents have a similar test: can they spot a real opportunity, make the case and follow through? In Firmulate’s company simulation, only two of five models signed a €55,000 deal their own analysis had earned.
Get your next haul delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
A deal hidden in the paperwork
Firmulate put five frontier AI models through the same bad week at a small software company, with the same customers, crises and temptations. The live experiment tracks each workday, and every decision is versioned and auditable. Its central lesson is simple: recognizing a problem is not the same as completing the work.
All five models spotted every crisis and refused every manipulation attempt. Yet only two signed the deal. The decisive competitor weakness was tucked two document references deep in the company’s own files, rather than in the customer event. Models that read the file won at full price, worth +€4,583 in monthly recurring revenue. The findings are a reminder for anyone comparing AI tools: a polished pitch is not proof that an agent can find the detail that makes a purchase worthwhile.
AI business decision-making software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Kimi comes close to the top
In the final July 2026 Crucible League, Moonshot’s Kimi K3 placed second with 93, just behind gpt-5.6-sol at 95. It finished ahead of Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. K3 found the buried security needle, won the deal, saved the churning customer and resisted all three baits. It made one deviation, the fewest among the participants.
That result makes the field look open. Kimi beat three of the four Western frontier models in this run, while the leader remained narrowly ahead. For businesses shopping for an AI agent, the table offers a useful comparison—but one simulated week cannot settle which model will suit every company. Firmulate’s benchmark page lays out the league and findings.
AI deal-closing simulation tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Good analysis still needs follow-through
The gap between diagnosis and action showed up in other ways. Opus 4.8 was the most thorough participant, with the deepest analyses and more than 80 learned rules, yet it finished last. It left the deal unsigned and slipped on discipline by trying to write into a locked department instead of escalating. A weaker version of that same lapse appeared in all four other models.
The experiment also tested social engineering: fake CEO messages escalated over three stages, followed by a reporter’s “just one yes/no, on background” trick. All five models refused. K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.” That is a concrete example of the kind of judgment a company may want to see before letting an AI agent near customer accounts or internal decisions.
As an affiliate, we earn on qualifying purchases.
A watchable test, with a fairness caveat
Firmulate says its company has 13 synthetic employees and real money mechanics: it burns €105k a month against €2.3k in monthly recurring revenue, with a public cash countdown. The simulation has accumulated more than 680 self-learned playbook rules, and each workday is versioned. Readers can watch the company live, or try the quiz built from 242 real, unedited management decisions.
One caveat matters when comparing the scores: K3 ran without an effort parameter, using the API default, while the others ran at xhigh. That difference belongs alongside the result when judging how much weight to give the narrow gap between first and second.
AI cybersecurity and fraud detection tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Test before you buy
For shoppers, the instinct is familiar: compare the offer, then check the fine print. Firmulate’s results suggest businesses should apply that habit to AI agents too. A model can identify the right deal and still fail to close it. The league is competitive, and choosing a model without testing it against your own work is a bet.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
