Squiz Series: AI agents and the law

Opening summary

The hidden cost of AI adoption is not regulatory lag or technical risk. It is the gradual erosion of human critical thinking, a second-order effect that compounds right when professionals need it most. Peter Lee, a partner at Simmons and Simmons, describes a system where convenience creates dependence, dependence atrophies judgment, and the organizations that resist this trade-off will own the long-term advantage. Policy leaders, corporate strategists, and knowledge workers alike should pay attention: the competitive edge will not come from deploying agents faster but from preserving the thinking those agents are designed to replace.

Key insights and analysis

The fragility under the convenience

When most teams evaluate AI, they ask: How fast can we deploy? Lee suggests a different starting question: What happens when the infrastructure disappears? He describes the global AI stack as "fragile" because it depends heavily on a small number of US-based companies. The hidden consequence is not technical, it is a sovereignty trap. Australia, like many nations, relies on infrastructure it does not control. Lee's advice is direct: "really understand the tech stack that they are operating with... understand where the fragility... and their reliance on overseas companies are, where your data sovereignty lies." The immediate comfort of plugging into a powerful API creates a downstream exposure that only becomes visible during a crisis. Organizations that map these dependencies now, rather than after an outage, buy themselves years of resilience.

"Agents have got a level of autonomy that we haven't seen before from technology."

-- Peter Lee

The many hands problem and the coming litigation wave

Accountability in AI is a systems thinker's nightmare. Lee names the "many hands problem": frontier model companies, system developers, deployers, users, each in different jurisdictions with different laws. The conventional regulatory approach, the UK and Australia sticking with existing laws, creates confusion. Lee notes that "the only winners are the lawyers." But the non-obvious insight is that this patchwork is not permanent. Lee predicts a wave of litigation over the next five years: "You might get actions from trade unions, you might get class actions at an environmental level from communities." The implication is stark. Companies that wait for legislation will be shaped by court rulings they did not prepare for. The smart move is not compliance-watching; it is building internal governance now that can survive any legal landscape. Lee points to Microsoft's runtime governance model and his own idea of "agent resumes," treating agents like digital workers with defined roles and performance tracking. This is not just risk management. It is a structural investment that pays off when the litigation dust settles.

"Our companies, organisations of any stripe, sort of responsible for what an AI might do... this is a very complicated complex question and I think the law in most, in all countries I don't think is settled on this."

-- Peter Lee

The thinking recession nobody is preparing for

This is where Lee's analysis cuts deepest. The obvious threat to professional work is efficiency loss or job displacement. Lee argues the real damage is subtler: "the impact of this technology on our critical thinking skills." He calls it outsourcing your brain, the same mechanism that made Google Maps erode navigation skills, now running on cognitive tasks in law, consulting, and elsewhere. The danger compounds over time because each reliance makes future independent thinking harder. Lee's antidote is counterintuitive. He has stopped hiring data scientists and started hiring philosophers: "Top of our Recruit list were Computer Scientists, Data Scientists. Now I'm looking to hire philosophers." That is not nostalgia; it is systems-level reasoning. Data scientists optimize models; philosophers question outputs. In a world where models hallucinate and bias leaks through the cracks, the skill that matters is judgment, not computation. The professionals who thrive will treat AI as a sparring partner, not a substitute. That requires a deliberate, uncomfortable habit of thinking against the tool's output rather than accepting it.

The output

Immediate actions (next quarter):

• Audit your organization's dependence on foreign AI infrastructure. Map data sovereignty and identify single points of failure in your tech stack.
• Start writing "agent resumes" for any AI agent you deploy. Define its role, permissions, performance metrics, and kill criteria. This creates governance documentation that will be invaluable when liability questions arise.
• Replace one-hour AI-automated tasks with manual execution once a week. Rebuild the cognitive muscle your team is outsourcing.

Longer-term investments (6-18 months):

• Shift hiring criteria toward critical thinking: philosophers, ethicists, or anyone trained to challenge assumptions. This is an 18-month play that others will not copy quickly.
• Implement runtime governance for agentic AI, monitoring for "drift" in real time. Most teams will not pay for this until after a failure. Be the one who does it before.
• Lobby for sector-specific guidelines rather than waiting for general AI legislation. Clarity attracts talent and investment; ambiguity repels them both.
• Build internal training programs that treat AI as a sparring partner rather than a tool. This directly counteracts critical thinking erosion and creates a workforce that can verify AI output, not just consume it.