The AI Bubble: Why We Are Being Engineered as Tools, Not Users

The current AI boom is not a technological revolution. It is a massive, capital-intensive exercise in power consolidation. Cory Doctorow argues that the true purpose of modern AI deployment is the creation of reverse centaurs: human workers conscripted to serve machines rather than the other way around. By mapping the incentives of capital allocators against the reality of operational degradation, we reveal a system designed to bypass human expertise and accountability. For the reader, understanding this dynamic provides a clear advantage: the ability to distinguish between genuine productivity tools and systems designed to extract labor through accountability sinks. This is not just a critique of software. It is a warning about the structural instability created when we replace human agency with automated processes designed to maximize speed at the expense of human flourishing.

The Logic of the Reverse Centaur

In traditional automation theory, a centaur is a human augmented by a machine, like a cyclist or a user of a spell-checker. The machine expands human capability. Doctorow identifies a dangerous inversion: the reverse centaur. Here, the machine is the primary driver, and the human is the bottleneck, forced to keep pace with an assembly line that has no slack.

This is not an accidental byproduct. It is the design goal. When management views workers as obstacles to efficiency or sources of friction, AI becomes the perfect tool to replace them with automated agents that never argue, never push back, and never provide ego-destroying corrections.

A reverse centaur is someone who is conscripted to help the machine... You are Lucy and Ethel trying to get chocolates off the assembly line and get them into the chocolate box and the boss is running the assembly line at the maximum speed that the humans can conceivably do it at.

-- Cory Doctorow

Accountability Sinks and the Erosion of Recourse

The most insidious consequence of this shift is the accountability sink. When companies replace human service agents with AI, they gain a convenient scapegoat for system failures. If an AI provides incorrect information, such as the Air Canada chatbot that promised a bereavement fare refund that the company later denied, the machine takes the blame. This effectively shields the corporation from the consequences of its own poor design.

This creates a feedback loop. Because the company holds market power, they have no incentive to improve the service. The AI functions as a barrier between the customer and the company, allowing the firm to say that they do not care and do not have to. Over time, this leads to the automation blindness observed in high-stakes environments. When a human is relegated to merely rubber-stamping AI output at high speed, they lose the ability to maintain the vigilance required to catch the machine's errors.

It does not matter if it is wrong, it just matters if it does not produce more of a loss than a gain and there is a certain perverse incentive to give people that advice that costs the money if there is no way for them to get the money back.

-- Cory Doctorow

The Material Limits of the Bubble

The AI bubble is significantly larger than previous tech cycles, with 1.4 trillion dollars in projected infrastructure spending by 2027. Doctorow points out that this is not just digital speculation. It requires massive material inputs, including energy, grid capacity, and complex hardware manufactured in specific global hubs.

Conventional wisdom suggests this growth is inevitable, but Stein's Law dictates that anything that cannot go on forever will eventually stop. The danger lies in the austerity response that historically follows such collapses. When the bubble pops, the resulting economic instability often drives populations toward authoritarian solutions. The competitive advantage for the reader lies in recognizing that these systems are not inevitable. They are choices made by oligarchs who prioritize solipsistic control over functional, human-centric design.

Key Action Items

• Audit your tools for Reverse Centaur dynamics: Over the next quarter, evaluate whether your software tools are augmenting your output or forcing you to perform data entry for the machine's benefit. If the latter, look for alternatives that prioritize user control.
• Identify Accountability Sinks in your workflow: Within the next 30 days, map where AI or automated systems are currently making decisions that affect your team or customers. Determine who is ultimately responsible when these systems fail.
• Invest in human-in-the-loop safeguards: If you are deploying AI, ensure human oversight is not just a rubber-stamp process. Design workflows where the human is expected to catch errors, not just confirm them. This pays off in 12 to 18 months by preventing systemic failures.
• Shift from growth at all costs to material utility: When evaluating new tech investments, ignore the hype cycle. Focus on whether the tool produces real-world value that is not dependent on massive, unsustainable capital subsidies.
• Advocate for structural separation: Support policies that prevent the concentration of power, such as antitrust enforcement and the separation of investment and commercial banking. This is a long-term investment in market stability that pays off over years.