Institutionalizing Experimental Failure to Drive Long--Term Innovation
The American Experiment: Why Science and Democracy Require Failure
The founding of the United States was a deliberate, large-scale scientific experiment. By writing the promotion of science into the Constitution, the founders built a system that valued empirical proof over dogma. The true advantage of the American project was its structural commitment to free thinking. The founders were willing to test, fail, and iterate on their governance just as natural philosophers tested theories of gravity or electricity. For modern leaders, the lesson is clear: innovation requires the same tolerance for public failure and the same reliance on rigorous protocols that Benjamin Franklin and his peers used to turn parlor tricks into modern science. Organizations that treat themselves as static structures will be outpaced by those that treat themselves as experiments.
The Hidden Cost of Parlor Trick Innovation
Benjamin Franklin succeeded because he was committed to documentation. As Dr. Robert Allison notes, Franklin turned electricity from a spectacle into a science by recording protocols that allowed others to replicate his work. This creates a specific dynamic: when an innovator documents their process, they invite scrutiny. When they do not, they remain a parlor trick artist.
Franklin is experimenting with these things, along with a group of other guys who are doing things. And he writes down the protocols for everything they are doing so they can be replicated. So he takes electricity which had been a parlor trick and turns it into a science.
-- Dr. Robert Allison
The downstream effect of this transparency was profound. By publishing his work in Europe, Franklin forced the scientific community to engage with his ideas, even when he was geographically isolated. In modern terms, Franklin’s paper trail was his moat. It allowed his work to survive his absence, creating a feedback loop where European scientists validated his theories and built his reputation for him.
How Systems Route Around Your Solution
The 1755 earthquake in Boston shows how systems and the people within them react to new data. When Professor John Winthrop used evidence, such as the fall patterns of bricks, to argue that earthquakes were undulatory waves, he faced immediate pushback from Reverend Prince, who claimed the disaster was caused by the proliferation of lightning rods.
This is the classic blame the innovation response. Because lightning rods were a visible change to the environment, they became a convenient scapegoat for a phenomenon people did not understand. Winthrop’s challenge was not just scientific; it was social. He had to dismantle a theory that provided comfort to those who feared change. The lesson is that when you introduce a technical solution, you are also introducing a new variable that will be blamed for every unrelated failure that follows.
The 18-Month Payoff of Tinkering
The 1761 and 1769 expeditions to observe the transit of Venus represent a rare moment where global cooperation was driven by the scarcity of the event. Because the transit would not recur for over a century, the stakes were absolute.
He is so overwhelmed by the thought that he is going to see Venus transiting across the sun and no one will see this again for about 150 years. He faints.
-- Dr. Robert Allison
This was not efficient work. It required massive logistical effort, such as transporting 24-foot telescopes to remote locations like Newfoundland, for a single data point. Yet, this effort provided the scale of the solar system. In contemporary business, we often prioritize low-hanging, repeatable metrics. But the most durable advantages come from tinkering on problems where the payoff is delayed by years or decades. Most organizations will not commit to a project with a 150-year feedback loop; that is why the few that do, win.
The Limits of Experience vs. Theory
The debate over smallpox inoculation in 1721 reveals the friction between established authority and empirical evidence. Cotton Mather, citing both the Royal Society and the lived experience of an enslaved man named Onissimus, advocated for inoculation. The local authorities, fearing public panic, opted for the performative action of sweeping the streets to appear busy.
The system responded with hostility: a bomb was thrown through Mather’s window. This demonstrates a recurring pattern where the obvious solution, such as doing nothing or performing busywork, is preferred over the effective solution because the latter requires acknowledging a terrifying reality. As John Dickinson famously noted, experience must be our guide. Reason may mislead us. The founders understood that theories are dangerous until they are stress-tested by reality.
Key Action Items
- Audit your parlor tricks: Identify projects that rely on individual intuition rather than documented protocols. Over the next quarter, force these to be written down so they can be replicated or debunked.
- Create a failure budget: Like the founders who viewed the Constitution as an experiment, treat your organizational processes as hypotheses. If a process is not producing results, be prepared to replace it in 6 to 12 months.
- Seek out Venus Transits: Identify one long-term, high-stakes project that your competitors are ignoring because the payoff is too far out. Invest in it now. This pays off in 5 to 10 years.
- Map your scapegoats: When introducing a new tool or strategy, anticipate what people will blame when things go wrong. Pre-emptively communicate why the new system is not the cause of unrelated failures.
- Prioritize tinkering culture: Encourage employees to work with their hands or build prototypes, like Jefferson’s polygraph. This creates a deeper understanding of mechanics that abstract planning cannot replicate.
- Document the why, not just the how: When you make a decision, record the theory behind it. In 18 months, revisit the record to see if the theory held up or if reality forced a pivot.