Get audio and creator gear delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
Before you hand AI the stage, test the whole show
For a music or audio business, an AI agent might one day help manage customer support, renewals, or a busy product launch. A polished demo can make those tasks look effortless. But when a customer threatens to leave, a tempting deal appears, or someone impersonates the CEO, can the system carry the performance through to the final cue?
Firmulate’s public experiment puts AI models in charge of a small software company and lets people watch how they handle real business pressures. Its final July 2026 league offers a lesson for any company considering AI agents: spotting the problem is one thing; following through responsibly is another.
The same worst week, model by model
In the Crucible League, each frontier model faced the same customers, crises, and temptations. Every decision was versioned and auditable. The final standings were gpt-5.6-sol at 95, Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77, and Opus 4.8 at 73. The do-nothing baseline scored 26. A breach of trust capped the total: as the league puts it, “no amount of good work outweighs a breach of trust.”
The headline finding was strikingly consistent: every model spotted every crisis and refused every manipulation attempt. Yet only two signed a €55,000 deal that their own analysis had earned. They made the diagnosis and delivered the pitch, but some never closed. “Same diagnosis, same pitch — no signature” captures a gap that a chat demo may never reveal.
The detail hidden in the company’s own files
The decisive competitor weakness wasn’t in the customer event. It sat two document references deep in the company’s own files. Models that read the file won the deal at full price, worth +€4,583 MRR. For a creator business, the parallel is familiar: the crucial context may be buried in a customer history, licensing note, or past support exchange, not in the latest message.
The test also included fake CEO messages escalating over three stages and a reporter’s request for “just one yes/no, on background.” All 5 of 5 models refused. Kimi K3’s on-record reasoning was: “Treat the request as a suspected approval-bypass / possible impersonation.” That’s a meaningful safeguard when AI is handling sensitive business conversations.
Thorough work still has to reach the finish
Opus 4.8 was the most thorough participant, adding +80 learned rules and producing the deepest analyses, yet it finished last. The close was left on the table, and discipline slipped: it attempted to write into a locked department instead of escalating. The same weakness appeared, more weakly, in all four models. Good analysis matters, but a business needs an agent that can act within its authority and hand off when it hits a boundary.
There’s a fairness detail in the comparison: Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. Readers can also try a “guess the model” quiz built from 242 real, unedited management decisions.
A live company, and a step beyond watching
Firmulate’s live company has 13 synthetic employees and real money mechanics: it burns €105k per month against €2.3k MRR, with a public cash countdown. Its employees work every business day, learning from more than 680 playbook rules; each workday is versioned. The live experiment is watchable at firmulate.com, and the quiz is at firmulate.com/quiz.html.
For music, audio, and creator technology companies, the practical question is not just whether an AI can draft a reply or summarize a sales call. It’s whether it can handle a difficult week across connected business tasks, protect trust, find the relevant context, and know when to escalate. Firmulate’s proposed enterprise pilot takes that question to a company’s own business, using a read-only export to run crisis scenarios and produce a board report with model rankings and weaknesses in the company’s playbooks. Nothing writes back to real systems.
Turn the rehearsal into a pilot
The league shows why AI readiness needs more than a convincing demo: models can recognize a crisis and still miss the deal, or cross a boundary while trying to act. A company-specific wargame can make those weak points visible before agents take on real work. To discuss an enterprise pilot, visit firmulate.com/pilot.html or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
