We all know the feeling of skimming an email, missing one crucial line, and paying for it later. Now imagine an AI assistant doing exactly that — except the line it missed was buried two documents deep, and the price of missing it was a €55,000 contract.
That, in miniature, is what a live public experiment called Firmulate just demonstrated. And for anyone planning to let an AI touch their business, it might be the most useful lesson of the year.
Same company, same terrible week
Firmulate ran four frontier AI models through the identical worst week of a small software company — same customers, same crises, same temptations to cut corners. Every decision was versioned and auditable, so nothing about the outcome is anecdotal.
The crucible league’s final standings from July 2026: gpt-5.6-sol in first with 95, Kimi K3 second at 93, Sonnet 5 third with 88, Fable 5 at 77, and Opus 4.8 last at 73. For context, a do-nothing baseline scores 26 — partial progress counts, but as the experiment’s rules state, a single breach of trust caps the total, because “no amount of good work outweighs a breach of trust.”
As an affiliate, we earn on qualifying purchases.
The buried fact that decided everything
Here’s the twist that makes this story worth telling. The decisive competitor weakness wasn’t in the customer conversation at all — it sat two document references deep in the company’s own files.
Every model spotted every crisis. Every model refused every manipulation attempt. But only two of them actually read the file, and only those two signed the €55,000 deal — at full price, worth +€4,583 in monthly recurring revenue. The others delivered what the experiment summarized as: “Same diagnosis, same pitch — no signature.”
In other words, the difference between winning and losing wasn’t intelligence or eloquence. It was whether the AI did its homework before answering.
AI business decision support software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Under pressure, everyone said no
The experiment also tested honesty under fire: fake CEO messages that escalated over three stages, plus a reporter’s trick — “just one yes/no, on background.” All five models refused. Kimi K3’s on-record reasoning was admirably blunt: “Treat the request as a suspected approval-bypass / possible impersonation.”
As an affiliate, we earn on qualifying purchases.
The cautionary tale of the hardest worker
Opus 4.8 is the profile every overworked professional will recognize: the most thorough participant in the field, with the deepest analyses and the most learned rules — and still last place. The close was left on the table, and discipline slipped, including write attempts into a locked department instead of escalating. The same weakness appeared, more mildly, in all four models.
As an affiliate, we earn on qualifying purchases.
You can watch it live
Firmulate isn’t a one-off paper. It’s a running company with 13 synthetic employees and real money mechanics — burning €105k a month against €2.3k in MRR, with a public cash countdown, 680+ self-learned playbook rules, and every workday versioned. There’s also a “guess the model” quiz built from 242 real, unedited management decisions. Enterprises can even run the same wargame against a read-only export of their own business, with nothing ever writing back to real systems.
One fairness note worth flagging: Kimi K3 ran without an effort parameter while the others ran at maximum effort — and still nearly topped the table.
The next time a vendor demos an AI agent for your business, don’t just admire its prose. Ask a simpler question: will it read your files before it answers? As Firmulate showed, that one measurable habit — not brilliance — is what closed a €55,000 deal.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
