🔍 Read the full analysis: The Curious Case Of AI Managers Getting 26 Points In Tough Tests on ThorstenMeyerAI.com

Prime Big Deal Days · Oct 6–7Offer from Amazon

Get the latest gadgets delivered free — and shop member deals

• Fast, free delivery on millions of items
• Access to Prime Big Deal Days deals on October 6–7
• Prime Video, Amazon Music and more included

Start your free Prime trial →Free trial for eligible customers · Cancel anytime

As an affiliate, we earn on qualifying purchases.Hide

In a recent benchmark conducted by Firmulate, four frontier AI models managing a simulated company during a week of crises scored between 73 and 95 points. For more details, see the original analysis. The lowest score, 26 points, was assigned to a do-nothing baseline, highlighting the benchmark’s focus on minimal viable management and trustworthiness under pressure. This development raises questions about the readiness of AI for enterprise management roles and the reliability of current models in high-stakes environments.

The benchmark, known as the ‘Crucible League,’ tasked AI models with managing a small software firm during seven days of simulated crises, including customer issues, social engineering, and trust attacks. This approach reflects ongoing efforts to evaluate AI reliability in enterprise scenarios, as detailed in the original analysis. The models’ decisions were fully auditable, and their scores reflected their ability to handle real-world business challenges while maintaining trust and integrity.

The top performer, gpt-5.6-sol, scored 95 points, followed closely by Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77, and Opus 4.8 at 73. The scoring system included a baseline of 26 points for minimal effort, emphasizing that partial progress is valued but trust breaches are heavily penalized. Insights from this benchmark are discussed in the original analysis. Notably, no model achieved a perfect 100, as the benchmark designers consider such a score suspicious, implying unmeasured or unearned perfection.

One of the key findings was the models’ ability to identify crises and refuse manipulation attempts, such as social engineering attacks. For example, all models refused a staged CEO impersonation, demonstrating a basic level of trustworthiness. However, the models’ effectiveness varied significantly in executing follow-through tasks, such as closing sales deals or reading internal documentation. Only two models successfully identified critical information buried in company files, enabling them to secure a €55,000 deal—an outcome that distinguished the most capable models from the rest.

Implications for AI in Business Management

The results highlight that AI’s ability to manage complex, high-pressure business scenarios depends not only on communication skills but also on thoroughness, trustworthiness, and follow-through. While current models can recognize crises and refuse manipulation, their capacity to read and utilize internal documentation effectively remains inconsistent. This raises concerns about deploying AI in real enterprise environments where trust, integrity, and comprehensive decision-making are critical.

For organizations integrating AI agents into customer support, CRM, or decision-making, these findings underscore the importance of evaluating not just language proficiency but also operational reliability, task completion, and ethical behavior. The benchmark’s transparent scoring and auditable decision trail provide a valuable tool for assessing AI readiness in management roles.

Background of AI Management Benchmarks

Traditionally, AI performance benchmarks focus on language capabilities or task-specific accuracy, often overlooking the broader management context. The ‘Crucible League’ is among the first to evaluate AI models on their ability to handle real-world business crises, including social engineering, trust maintenance, and decision execution under stress. The benchmark was designed to simulate a week’s worth of business challenges, with full transparency and auditable decision logs to ensure fairness and accountability.

Previous assessments of AI in enterprise roles have highlighted limitations in consistency and reliability, but few have tested models in a scenario that measures their capacity to manage trust and follow-through. The July 2026 results mark a significant step toward understanding AI’s practical management potential and its current shortcomings.

It is important to note that the scores reflect not just language or superficial decision-making but also the models’ ability to read internal documentation, refuse manipulative requests, and uphold trust—key factors in real-world AI management applications.

Unresolved Questions About AI Management Performance

It is not yet clear how these benchmark results will translate to real-world enterprise environments, where variables are more complex and stakes higher. The models’ performance in simulated crises may differ from actual business management, especially regarding long-term trust and ethical decision-making. Additionally, the impact of different configurations, training data, and operational parameters on these scores remains to be studied.

Furthermore, the benchmark’s design intentionally caps scores and does not measure the full spectrum of AI management skills, such as strategic planning or emotional intelligence. Whether future models will surpass current limitations is still uncertain.

Next Steps for AI Management Benchmarking

Researchers and developers are expected to refine AI models to improve reading internal documents, follow-through, and trustworthiness. Future benchmarks may incorporate longer-term management scenarios and more complex decision-making tasks, aiming to better reflect real-world enterprise needs.

Organizations interested in deploying AI for management should monitor these developments and consider participating in or simulating similar tests to evaluate their AI tools’ readiness. The ongoing evolution of benchmarks like the Crucible League will help set industry standards for trustworthy AI management.

Additionally, transparency and auditable decision logs, as used in this benchmark, are likely to become standard features in enterprise AI solutions, fostering greater accountability and reliability in AI-driven management.

Source: ThorstenMeyerAI.com