Improving Accuracy by Replacing Proxy Data with Direct Observation

In this conversation, marine conservation biologist Katherine McDonald explains that our understanding of shark populations is limited by a reliance on flawed data collection. By moving from fishery-dependent data, which fluctuates based on economic and regulatory incentives, to independent monitoring, scientists can see the true trajectory of marine health. The consequence of this shift is the realization that many recovered populations are simply the result of increased detection, while others remain in long-term decline. This offers a clear advantage: when you stop relying on convenient, proxy-based metrics and prioritize direct, ground-truth observation, you can distinguish between noise and genuine systemic change, allowing for more precise, long-term strategic investments.

The Illusion of Visibility and the Trap of Proxy Data

Most organizations, whether in conservation or business, fall into the trap of using proxy data to measure success. As McDonald explains, relying on fishery data to track shark populations is skewed because the effort and method change alongside the value of the fishery. When regulations shift or mesh sizes change, historical data becomes incomparable to current data. This creates a feedback loop where managers believe they are seeing trends, when they are merely observing the artifacts of their own changing collection methods.

"If you have been fishing within that fishery for decades with a net that has a particular size mesh and then NOAA decides that to avoid bycatch or because it is going to be more effective. They want to change the mesh size, all of a sudden your past data and your future data cannot accurately be compared anymore because your method has changed."

-- Katherine McDonald

The lesson is that standardizing your input metrics is a prerequisite for any meaningful analysis of change. If your measuring stick evolves alongside the system you are tracking, you lose the ability to establish a baseline.

The Downstream Cost of Passive Symbiosis

Systems thinking requires us to look past surface-level interactions to see the potential for hidden friction. The traditional view of the Remora-Manta Ray relationship, that it is a benign, mutually beneficial symbiosis, is being challenged by McDonald’s team. By observing Remoras entering the cloaca and gills of Manta Rays, the team is identifying a transit opportunity that may actually be a parasitic or disruptive burden on the host.

"Almost all of the literature around these relationships between Remoras and their hosts, assume that their hosts are unbothered by that relationship. That it maybe even benefits the host... This little brick contributes to a section of the wall that is asking questions about whether Remora host relationships are really as problem free for the host, as has been previously assumed."

-- Katherine McDonald

This suggests that many stable systems are held together by assumptions of non-interference. When we look closer, we often find that what appears to be a neutral interaction is actually a source of hidden, compounding costs for the primary actor.

Why Good Design Outlasts Innovation

McDonald reframes the evolutionary history of sharks not as primitive, but as a series of highly successful design choices made early on. In systems, there is a tendency to mistake new for better. However, as McDonald notes, if a design works exceptionally well, it does not need to change. This is a counter-intuitive insight for technical practitioners: the most durable systems are often those that solved their core problems early and effectively, rather than those that constantly iterate for the sake of novelty.

Key Action Items

• Audit your metrics for Methodological Drift: Over the next quarter, review your primary KPIs. If your collection method or definition of success has changed, acknowledge that your historical baseline is likely broken and recalibrate.
• Challenge Benign Assumptions: Identify the Remoras in your system, the processes or partners that are assumed to be harmless or beneficial. Investigate if they are actually creating hidden, downstream friction for your primary operations.
• Prioritize Direct Observation: When data feels ambiguous, stop relying on proxies. Invest in fisheries-independent equivalents, direct, hands-on audits or primary research, to verify what is actually happening in your environment. This pays off in 12-18 months by preventing strategic errors based on skewed data.
• Focus on Systemic Upstream Health: Long-term results in complex systems often depend on factors outside the immediate scope. For sharks, it is water quality; for your organization, identify the upstream conditions like culture, infrastructure, or talent that determine success, even if they are not the primary focus of your daily tasks.
• Embrace Ancient Design Principles: Evaluate where you are over-engineering. If a core process is stable and functional, resist the urge to innovate it. Focus your energy on where the system is truly failing, not where it is already performing optimally.