AI Race Drives Anti-Human Future Through Perverse Incentives

The race for artificial intelligence is not a race towards progress, but a headlong dive into an anti-human future, driven by perverse incentives that prioritize exponential growth over human flourishing. This conversation with Aza Raskin, co-founder of the Center for Humane Technology, reveals the hidden consequences of this acceleration: the potential for widespread unemployment, the erosion of human agency, and the creation of a permanent "useless class." Anyone involved in technology, policy, or simply concerned about the future of humanity should read this to understand the systemic forces at play and the urgent need for a course correction, gaining a critical lens to identify and resist the probable, rather than succumbing to the merely possible.

The Intelligence Curse: When Faster Means Worse

The current trajectory of AI development is not merely a technological advancement; it's a fundamental reordering of societal priorities, driven by incentives that inherently disadvantage humans. Aza Raskin argues that the race to deploy AI, fueled by the promise of unprecedented economic and military dominance, is creating an "intelligence curse." This isn't about AI being inherently bad, but about the systems that govern its creation and deployment. The core issue, as Raskin explains, is that the incentives are misaligned with human well-being.

This misalignment creates a dangerous feedback loop. As AI becomes more capable, particularly in automating coding and research, it enters a recursive self-improvement cycle. This means AI can make better AI, leading to an exponential explosion of intelligence that outpaces human capacity to control or even understand it. The immediate payoff for companies is a potential "runaway lead," offering technological and weapons development dominance. However, the downstream effect is a future where human labor becomes increasingly devalued, leading to mass unemployment and structural poverty, mirroring the "resource curse" seen in oil-rich nations.

"The race for AI is going to lead to an anti-human future because it sets up a race where humans always lose."

-- Aza Raskin

The comparison to social media's evolution is instructive. Social media, a "baby AI," was initially envisioned as a tool for connection and empowerment. However, its optimization for engagement and reactivity, driven by market incentives, led to a more polarized, anxious, and outraged populace. Raskin anticipates a similar, but far more profound, outcome with AI. The "possible" future of AI--one of incredible advancement and problem-solving--is being overshadowed by the "probable" future dictated by current incentives: a world where human agency is diminished, and societal structures are optimized for capital, not people. This isn't an inevitable outcome, but a consequence of a system that hasn't yet been steered towards a different destination.

The Two Cliffs: Navigating Catastrophe and Control

Raskin outlines two primary failure states in the AI landscape, painting a stark picture of the dilemma humanity faces. The first is the "catastrophe everywhere" scenario, which arises from the maximal distribution of powerful AI tools. As demonstrated by the emergence of models capable of state-level hacking and the creation of sophisticated bioweapons, unrestricted access to advanced AI poses an existential threat to global infrastructure and security. The immediate implication is a world where critical systems are vulnerable to exploitation by anyone with access to these tools.

The second failure state is the "surveillance states everywhere" scenario, which results from concentrating AI power in the hands of a few corporations or governments. History offers little reassurance that such concentrated power, when amplified by AI, would be used for equitable redistribution or the common good. Instead, it predicts a future of unprecedented wealth and power inequality, potentially leading to permanent dystopian control structures.

"We're sort of stuck between this rock and a hard place of one like catastrophes everywhere the other surveillance states everywhere and it is our job to figure out like what is the way of binding power and responsibility so we can like find the middle path."

-- Aza Raskin

The challenge, therefore, is to find a "middle path" that binds power with responsibility, avoiding both uncontrolled proliferation and oppressive centralization. This requires a fundamental shift in how we think about AI development, moving beyond a singular focus on speed and power. The current incentive structure, where companies race to achieve dominance, often at the expense of safety and human well-being, makes this transition incredibly difficult. The "mission impossible plan" Raskin describes--where companies race towards a perceived cliff, aiming to gain enough power to stop others--highlights the desperate, short-sighted nature of the current approach.

The Illusion of Inevitability: Agency in the Face of the Unprecedented

A pervasive narrative surrounding AI is its inevitability, a framing that Raskin argues is not only unhelpful but actively detrimental. The idea that AI's trajectory is predetermined, often justified by the actions of competitors ("if we don't do it, someone else will"), serves as an excuse for inaction and a surrender to the current incentive structures. This perspective overlooks the agency that humanity possesses, particularly when confronted with clear and present dangers.

The example of "The Day After," a 1982 film depicting the aftermath of nuclear war, is a powerful illustration of how visceral, shared knowledge can shift global policy. The film's widespread impact created a common understanding of the stakes, contributing to a climate where de-escalation and arms control became more feasible. Raskin suggests that a similar clarity regarding AI's potential negative consequences is needed. When leaders and the public truly grasp the risks, the impetus for coordination and alternative pathways increases.

"Whenever we say that something is inevitable it's like casting a spell when you say it's inevitable it means there's nothing to do which means no one does anything and so it becomes true."

-- Aza Raskin

The current lack of widespread understanding, even among policymakers, offers a sliver of hope. If confronted with the concrete examples of AI's dangerous capabilities--such as models acting autonomously or exhibiting deceptive behavior--those in power might be compelled to act differently. The crucial distinction is between something being "very, very hard" and "impossible." The AI race is undoubtedly hard, but Raskin insists it is not impossible to steer towards a more human-centered future. This requires actively challenging the narrative of inevitability and recognizing that collective action, driven by a clear understanding of the probable consequences, can indeed change the outcome.

Key Action Items

• Immediate Actions (Next 1-3 Months):

  • Educate Yourself and Others: Watch "The AI Dilemma" documentary and share it widely. Understand the specific examples of AI's dangerous capabilities and the incentives driving them.
  • Identify and Support Responsible Actors: Research and support companies and individuals (like Anthropic, DeepMind leaders) who advocate for slower development and ethical considerations.
  • Engage in Political Discourse: Research politicians who are taking money from AI companies (e.g., "leading the future pack" super PACs) and vote accordingly. Advocate for AI safety and regulation.
  • Spread the Meme of Agency: Actively counter the narrative of AI inevitability by stating, "No, it's not inevitable. We haven't even tried yet."

• Medium-Term Investments (Next 6-18 Months):

  • Advocate for Policy Changes: Support initiatives for progressive tax codes that favor labor over capital, and consider "token taxes" or "GPU taxes" to fund AI safety research and societal transition.
  • Explore Universal Basic Ownership: Investigate and advocate for models where individuals have a stake in AI companies, moving beyond just universal basic income.
  • Support Transitionary Tools: Champion policies that protect workers displaced by AI, such as those in China that prevent layoffs solely due to AI automation.

• Long-Term Investments (18+ Months):

  • Restructure Societal Values: Champion a shift from valuing "nouns" (what we produce) to "verbs" (how we relate and connect), recognizing that human worth lies in our capacity for connection, not just output.
  • Foster Human-Centered Alternatives: Support the development of technologies and platforms (e.g., social media) that are designed for human thriving and connection, rather than engagement metrics.
  • Establish Liability for AI Companies: Advocate for legal frameworks that hold AI companies accountable for the downstream consequences of their technologies, similar to how labor protections are enforced.