Generative AI Reshapes Product Development to an R&D Experimentation Model

The product development landscape has been fundamentally reshaped by generative AI, forcing a radical departure from traditional roadmap-driven methodologies. This conversation with Richard White, founder of Fathom AI, reveals that the core challenge is no longer just estimating effort and impact, but navigating a reality where both can change drastically in a matter of months, or even weeks. The implications are profound: companies must shift from predictable roadmaps to a more experimental, R&D-like approach, embracing a "Jenga model" of product development where experimentation and iteration are paramount. This offers a significant advantage to leaders who can adapt, providing them with a framework to build in an era of unprecedented technological flux and unlock novel sources of organizational insight. Those who fail to grasp this shift risk being left behind as the pace of AI innovation accelerates.

The R&D Imperative: Navigating the AI-Driven Product Development Shift

The traditional product development playbook, honed over years of predictable technical landscapes, is now largely obsolete. As Richard White explains, the advent of generative AI has inverted the very foundations of how software is built and managed. The core challenge for product managers has historically been estimating effort and impact, a task that, while complex, offered a degree of predictability. However, AI has introduced a level of volatility that renders these old methods insufficient. A project spec'd out today using an LLM might take six months to build, but waiting six months could see that same project achievable in six hours with a newer model, and with a significantly better output. This dynamic fundamentally breaks the linear progression of roadmapping.

"everything you knew about product managing kind of gets thrown out the window."

This necessitates a pivot from a roadmap-driven approach to one that mirrors Research and Development. Instead of meticulously planning every step, companies must foster an environment of continuous experimentation. White describes his team’s approach as akin to an R&D function, where exploration is not a bug, but a feature. This involves a deliberate separation of concerns: a dedicated AI team focuses on prototyping, exploring different models, and identifying novel use cases, essentially creating "specs" through functional prototypes. These prototypes are not intended for production but serve as high-fidelity blueprints for the core engineering team, which then handles the complex task of scaling, hosting, and serving these AI-driven features reliably. This division allows for rapid iteration and exploration without derailing the core operational stability of the product.

The Jenga Model: Experimentation in a Shifting Landscape

To manage this new reality, White introduces the "Jenga model" of product development. This metaphor captures the delicate, iterative process of exploring AI capabilities. Each "block" in the Jenga tower represents a potential model or use case. The team "pushes" on a block--tries a model with a specific use case--and observes the reaction. If it feels unstable or doesn't yield the desired results, the approach is to "find another block." This means either trying a different model for the same use case or exploring a different use case with the current model. This fluidity is crucial, as the landscape of AI models and their capabilities is constantly evolving. The key is to avoid getting stuck pushing a single block for too long, a lesson learned from the inherent instability of the Jenga tower.

"if you push on a block and it you know you get resistance our thing is find another block right that means that model is not the right tool for the job"

This experimental approach requires a tolerance for failure. White suggests that the AI team should be failing "50% of the time." This isn't an endorsement of inefficiency, but a recognition that true innovation in AI necessitates bold exploration. Many of these experiments are "hail marys"--long shots that might just work. The risk of failure is mitigated by the team’s profile: often earlier in their careers, comfortable with ambiguity, and adept at reading and evaluating research papers. This allows them to navigate new models and technologies more effectively. Furthermore, the team’s preference for a fully remote setup, with the AI engineering team opting for in-person collaboration, highlights the psychological benefits of tackling complex, uncertain problems together. This collaborative environment fosters resilience and shared learning, essential for navigating the inherent failures in AI development.

The "Taste" of AI: Qualitative QA in a Sea of Outputs

A significant downstream consequence of AI-driven development is the challenge of quality assurance. Traditional QA, focused on deterministic outcomes and binary success/failure metrics, is ill-equipped for the probabilistic nature of AI. White emphasizes that AI outputs often require a qualitative assessment, a "taste" that goes beyond automated checks. This "taste" is difficult to define but crucial for shipping high-quality AI features. It involves a deep understanding of the problem domain and a discerning eye for nuance in the AI's output. Larger, more conventional companies often struggle with this because they lack this inherent "taste," leading to mediocre AI features.

"I think that one of the reasons why all these large companies kind of aren't very good at shipping you know their traditional software aren't very good at shipping ai features because they have no taste and because they have no taste they just don't our qa this and so it spits out something and they ship it again"

Developing this "taste" requires a shift towards qualitative feedback loops. This might involve executives stepping into the QA process themselves, creating a culture where subjective evaluation is valued. The concept of "narrative as source code," proposed by Henrik, is a powerful framework for this. A strong company narrative provides a consistent guiding principle for decision-making, acting as a qualitative filter for features and outputs. For startups, this taste often resides with the founder; for larger organizations, it requires a deliberate effort to cultivate and empower taste-makers. This qualitative rigor is essential not only for internally developed features but also for evaluating third-party AI solutions. White's team, for instance, insists on 90-day pilots and rigorous testing plans before purchasing any AI product, a stark contrast to the "demo looked good, ship it" mentality of the past.

The Buy vs. Build Dilemma: Navigating the LLM Treadmill

The rapid pace of LLM development presents a significant strategic challenge: the "LLM treadmill." New models are released with such frequency that features built on older versions can quickly become obsolete or require substantial rework. Google's Gemini 3.0 release, for example, saw the rapid deprovisioning of capacity for Gemini 2.5, even before a stable API was available. This rapid cycle means companies could be forced to rebuild features every six to nine months, a pace that is unsustainable for many.

This reality has led Fathom AI to shift from a "build" to a "buy" strategy for many internal AI automations. The cost and effort of maintaining custom-built AI features on the treadmill often outweigh the benefits, especially for non-core functionalities. Instead, the focus has moved to becoming expert buyers of AI products, developing a rigorous process for evaluating vendors and ensuring quality. This involves not just assessing the initial demo but conducting thorough 90-day pilots with clear test plans and defined success metrics. The underlying principle is to make the buying process as frictionless as possible while maintaining a high bar for quality and reliability, recognizing that AI capabilities are now a commodity that can be purchased rather than solely built.

Actionable Takeaways for Navigating AI-Driven Development

• Embrace an R&D Mindset: Shift from rigid roadmaps to a culture of continuous experimentation and learning. Allocate resources for dedicated AI exploration teams.
• Implement the "Jenga Model": Encourage iterative testing of models and use cases. Prioritize flexibility over fixed plans, and be prepared to pivot when a particular approach proves unstable.
• Separate AI Prototyping from Core Engineering: Create distinct teams responsible for rapid AI prototyping and for the robust scaling and deployment of successful features.
• Develop Qualitative QA ("Taste"): Invest in processes that evaluate the nuance and quality of AI outputs, moving beyond purely automated testing. Empower individuals or teams to act as taste-makers.
• Master the "Buy" Decision: For non-core AI functionalities, become expert buyers. Develop rigorous vendor evaluation processes, including extended pilot programs and clear success criteria.
• Manage the LLM Treadmill: Be acutely aware of the rapid LLM release cycle. Prioritize building or buying solutions that are adaptable or where the cost of staying current is justifiable.
• Cultivate a Strong Company Narrative: Develop a clear, compelling narrative that guides decision-making and acts as a qualitative filter for AI-driven product development, especially when faced with abundant AI capabilities.