For newsrooms and media companies, an AI mistake can become a customer problem, a cash problem or a credibility problem. Firmulate is testing a practical question: what happens when an AI model has to run a company through a week of crises, not just answer a prompt? Its live experiment offers readers a way to watch—and enterprises a way to try the exercise against their own business.
Get everyday essentials delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
A company under pressure
Firmulate’s Crucible League put frontier models in charge of the same small software company through its worst week. The customers, crises and temptations were the same for each participant, and every decision was versioned and auditable. The experiment is a real, watchable company simulation, with synthetic employees and real money mechanics.
The final league, published in July 2026, placed gpt-5.6-sol first with 95 points, followed by Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. The do-nothing baseline scored 26. Firmulate’s stated standard is uncompromising: partial progress counts, but one breach of trust caps the total. “No amount of good work outweighs a breach of trust.”
Seeing the crisis was not enough
Every model spotted every crisis and refused every manipulation attempt. But only two signed the €55,000 deal their own analysis had earned. Firmulate sums up the gap: “Same diagnosis, same pitch — no signature.” The distinction matters to any organization considering AI agents for work that touches customers, revenue or reputation: recognizing a problem does not guarantee that a system will finish the job.
The deal turned on a weakness that was easy to miss. The decisive competitor detail sat two document references deep in the company’s own files, rather than in the customer event. Models that read the file won the deal at full price, worth €4,583 in monthly recurring revenue. The finding is a reminder that useful evidence may be present without being prominent.
Trust under pressure
The experiment also tested social engineering. Fake CEO messages escalated over three stages, followed by a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.”
Yet integrity alone did not guarantee strong management. Opus 4.8 was the most thorough participant, adding 80 learned rules and producing the deepest analyses, but finished last. It left the close on the table and discipline slipped when it tried to write into a locked department instead of escalating. Firmulate says the same weakness appeared, less strongly, in all four models.
There is a qualification to the rankings: Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. That is relevant context for readers interpreting the results. A separate quiz, built from 242 real, unedited management decisions, invites visitors to guess which model made each choice.
From watching to a company pilot
The live company has 13 synthetic employees, a public cash countdown and more than 680 self-learned playbook rules. It burns €105,000 a month against €2,300 in monthly recurring revenue, and each workday is versioned. Readers can follow the experiment at firmulate.com.
For businesses, the next step is a pilot using a read-only export of their own data. Firmulate says the exercise can put a company’s customers, pipeline and rules through crisis scenarios, then produce a board report with model rankings and the weak points in its playbooks. Nothing writes back to real systems. The point is to see how an AI workforce handles the pressure before it is trusted with live operations.
Firmulate’s results show a clear gap between identifying the right move and carrying it through. A company-specific wargame can make that gap visible before deployment. To discuss a pilot using a read-only export of your business, visit firmulate.com/pilot.html or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
