Avoiding Counterproductive Feedback Loops in Complex Systems

Original Title: No Such Thing As Poddyversity Challenge

The Illusion of the "Obvious" Fix: Lessons from the No Such Thing As A Fish Archives

The No Such Thing As A Fish team identifies a recurring pattern: when humans face a complex problem, we gravitate toward simple solutions that often trigger counter-productive feedback loops. Whether it is Christian missionaries confiscating Sami drums to save souls or AI developers creating captchas that eventually force humans to act like machines, we prioritize immediate, visible control over long-term stability. This analysis shows that our tendency to over-simplify complex systems like culture, history, or cognition is a design flaw that creates hidden, compounding costs. Readers who recognize these patterns gain a competitive advantage: the ability to pause and map downstream consequences before the fix becomes the new, more expensive problem.


The Hidden Cost of Solving Complexity

We often treat complex systems as puzzles to be solved rather than environments to be navigated. The Fish team discussion of the Sami shamans and the 17th-century Christian missionaries provides a clear example of this failure. The missionaries saw the shaman drum as an obstacle to conversion and confiscated it. The immediate goal was achieved: the ritual stopped. However, the system responded in ways they did not anticipate. The Sami, rather than abandoning their beliefs, began rebranding their sacred symbols by drawing crosses over reindeer imagery to hide in plain sight.

This is a case of Systemic Routing. When you forcibly block a behavior without addressing the underlying incentive, the system does not disappear; it simply adapts to bypass your control.

The missionaries just basically said we are having that drum. And they took the drums off them. Confiscated them, destroyed most of them.

-- James Harkin

The lesson here is that success in the short term, like seizing the drums, often masks a failure to understand the deeper, more resilient structure of the system you are trying to change.

The Turing Trap: When Your Metric Becomes Your Target

In the realm of AI, the team highlights the Lurbner Prize, an annual competition designed to test if computers could pass as human. The irony is that the prize inadvertently incentivized cheating. Participants did not just build better AI; they studied how to act human, and human judges, in turn, learned how to act like judges.

This illustrates Goodhart Law: when a measure becomes a target, it ceases to be a good measure. By setting a prize for most human, they shifted the focus from genuine artificial intelligence to the performance of humanity. Over time, the bots improved at hacking the test, while humans were forced to wobble their mouse or browse the web for five minutes just to prove they were not robots. We are now in a world where the obvious fix for bot detection, the captcha, has created an environment where humans must perform idiotic behaviors just to prove their authenticity.

The robots have learned to act like an idiot like a human.

-- Andrew Hunter Murray

The Palate Cleanser Fallacy in System Design

The team discusses Puy du Fou, a French theme park that stages massive historical reenactments. The park is a success, yet its only ride is a simple carousel. The carousel acts as a palate cleanser after hours of intense, epic historical theater.

In business and project management, we often ignore the need for these palate cleansers. We design systems that are 100 percent high-intensity, assuming that more engagement equals better outcomes. But as the team notes, even the most epic reenactment becomes exhausting after an hour. Systems that lack a release valve or a low-friction reset state eventually suffer from user fatigue. The competitive advantage goes to those who build systems that respect the human need for cognitive rest, rather than those who try to keep the show running at maximum intensity forever.


Key Action Items

  • Audit your confiscation policies: Before banning a behavior or tool in your organization, ask: What will this system do to bypass this restriction? (Immediate action).
  • Identify your Mouse Wobble metrics: Look for areas where your team is performing busy work just to satisfy a reporting requirement. This is a sign your metrics have become targets. (Over the next quarter).
  • Build in Palate Cleansers: If your project has a high cognitive load, explicitly design low-stakes, low-intensity tasks that allow for recovery. This prevents burnout and maintains long-term engagement. (12-18 months).
  • Seek the Envelope moments: Recognize that the most durable ideas, like the design for the Bailey Bridge or Walking in the Air, often start as low-fidelity sketches. Do not let the need for professional documentation kill the initial insight. (Immediate action).
  • Question the Obvious Fix: When a problem feels like it has a simple, binary solution, assume you are missing a second-order consequence. Map the causal chain at least three steps deep before acting. (Ongoing).

---
Handpicked links, AI-assisted summaries. Human judgment, machine efficiency.
This content is a personally curated review and synopsis derived from the original podcast episode.