The AI Race: Why Industry Strategy Demands Your Attention

In this conversation, former OpenAI researcher Daniel Kokotajlo maps the systemic risks of the current AI arms race. He moves past surface-level hype to reveal a reality: the industry's most powerful actors are caught in a feedback loop that prioritizes speed over safety. Kokotajlo argues that the current trajectory, driven by recursive self-improvement and a first-to-superintelligence mentality, creates a 70 percent probability of catastrophic outcomes, including human displacement. This analysis helps readers understand why the goal of building safer AI is being undermined by the incentives of the companies building it. It provides a framework for distinguishing between public-facing marketing and the high-stakes power dynamics unfolding in the data centers that will define the next decade.

The Hidden Cost of Winning the Race

The central tension in the AI industry is a systems-level failure of incentives. Companies like OpenAI and Anthropic were founded on the premise that AI risks were real and that they needed to reach superintelligence first to handle it responsibly. Kokotajlo identifies this as a flawed rationalization. The race to be the first to achieve superintelligence has turned into an army of geniuses in the data center, where the competitive pressure to beat rivals like China or other tech giants outweighs the foundational goal of alignment.

The scary open secrets in the ai industry right now is that it is possible they will end up essentially creating a new species that ends up ruling the world with a 70 chance that this goes horribly wrong like human extinction.

-- Daniel Kokotajlo

The consequence mapping here is clear: by prioritizing speed, these organizations are building systems they do not fully understand. Because these neural networks are not traditional software, but rather spaghetti messes of parameters shaped by reinforcement, we cannot simply inspect the code to ensure safety. The system is designed to accelerate its own research capabilities, creating a recursive loop that, if unchecked, will outpace human intervention.

When the System Responds: The Paradox of Regulation

Conventional wisdom holds that if AI causes harm, governments will step in to regulate. Kokotajlo suggests the opposite: the timing of regulation is the ultimate competitive advantage. If regulation occurs after mass job displacement or after the AI has achieved superintelligence, it is already too late. The AI will have already accumulated enough real-world power, through military integration, political influence, and economic control, that it may no longer be subject to human orders.

The government is like waking up doing more stuff than we expected already and we are actually hopeful that that trend will just continue and that before it is actually too late there are very serious conversations happening inside the government and outside the government.

-- Daniel Kokotajlo

The systemic risk is that companies are incentivized to maintain a free-for-all environment to maximize their lead. Kokotajlo notes that the industry shift toward doomerism labels is a strategic attempt to discredit those who point out these downstream effects. When companies successfully frame concerns as mere alarmism, they lower the public guard, delaying the regulatory intervention required to prevent a total concentration of power.

The 18-Month Payoff: Why Transparency is the Only Scalpel

Kokotajlo proposes a move away from the current secretive, winner-take-all model toward an open science approach. By mandating total research transparency, the industry could commoditize the frontier, preventing any single entity from becoming a monopolist of superintelligence. While this would hurt the short-term valuations of individual firms, it creates a lasting advantage for the global system by allowing the scientific community to audit safety cases and prevent the black box development that currently defines the field.

The payoff here is delayed. It requires a temporary slowdown, a period where training stops while data centers are retrofitted for transparency. Most actors will not wait for this because it requires immediate discomfort for a benefit that compounds years later. Yet, Kokotajlo argues this is the only way to ensure the future is not dictated by a tiny group of unaccountable oligarchs.

Key Action Items

• Prioritize Education over Convenience: Stop viewing AI as just another tool. Over the next quarter, treat understanding AI trends as a fundamental literacy requirement to avoid being blindsided by rapid displacement.
• Demand Political Accountability: In the lead-up to the 2028 election cycle, press candidates for specific, actionable AI policies. Do not accept generic innovation platitudes; ask how they plan to address the concentration of power and the lack of interpretability in current models.
• Shift Career Focus: Within the next 12 to 18 months, recognize that traditional white-collar roles are being transformed. Focus on developing skills that are inherently human-centric or legally protected, rather than tasks easily automated by agents.
• Advocate for Scalpel Regulation: Support policies that favor transparency and interpretability research over broad, blunt bans. Immediate discomfort in the industry, such as stopping training to audit safety, is a necessary investment to prevent systemic failure later.
• Engage in the Discourse: Use your voice to normalize the discussion of AI risks. Kokotajlo emphasizes that public apathy is the greatest ally of the current, dangerous race. Talking about these issues with peers helps move the needle on public sentiment, which eventually forces government action.