Enterprise AI Success Hinges on Proprietary Data, Not Commoditized LLMs

This conversation with Ali Ghodsi of Databricks and Arvind Jain of Glean reveals a critical divergence in how enterprise AI is perceived and implemented. While consumer AI has seen explosive adoption, the enterprise landscape is mired in a "fog of war," with a staggering 95% of AI projects reportedly failing. The non-obvious implication is that this failure rate isn't a sign of technological inadequacy but a necessary byproduct of genuine experimentation. The true advantage, as Ghodsi and Jain argue, lies not in leveraging readily available commodity models, but in harnessing proprietary data and complex internal workflows--the "secret sauce" that competitors cannot replicate. This piece is essential for CIOs, CTOs, and investors navigating the AI transition, offering a pragmatic framework for identifying durable advantage amidst the hype, and highlighting the strategic imperative of focusing on data and workflow integration rather than chasing the latest LLM trend.

The Commodity of Models, The Moat of Data

The current enterprise AI landscape is characterized by a curious paradox: while generative AI models like ChatGPT and Gemini have captured the public imagination and seen widespread adoption in personal and small-scale professional use, their translation into transformative enterprise solutions remains elusive for most. Ali Ghodsi, CEO of Databricks, points to the widely cited statistic that 95% of AI projects fail. However, he reframes this not as a failure of AI, but as a sign of healthy, albeit sometimes costly, experimentation. "When you are actually experimenting with new technology," Ghodsi explains, "if all of your projects are failing, that means you didn't just not trying enough." This perspective suggests that the high failure rate is a feature, not a bug, of exploring nascent technology.

The crucial insight here, and where conventional wisdom often falters, is understanding why these projects fail and where true value accrues. Both Ghodsi and Arvind Jain, CEO of Glean, emphasize that Large Language Models (LLMs) themselves are rapidly becoming commodities. "The LLM is a commodity," Ghodsi states plainly. "People are not saying that, but it is a commodity." He likens it to interchangeable gas stations; the primary differentiator becomes price and availability, not inherent superiority. This commoditization means that building a competitive advantage solely on the latest LLM is a losing proposition, as alternatives are readily available and quickly surpass each other.

The real differentiator, the enduring moat, lies elsewhere: in proprietary data and unique enterprise workflows. Jain articulates this clearly: "It's not about that [the LLM]. It's really comes down to your company, what data does your company have that's special that your competitors don't have? Can you leverage that and can you build AI that really understands that data? Because that's not a commodity." This is where the "secret sauce" of an organization resides--in the specific datasets, historical information, and intricate processes that define its unique operational landscape. Companies that can effectively integrate AI with this proprietary data and complex internal logic are the ones positioned for durable success.

"The LLM is a commodity. People are not saying that, but it is a commodity. Like we can get gas from this gas station, we can get gas from that gas station. It doesn't matter, just compare price."

-- Ali Ghodsi

This distinction is vital. Many companies are building "commodity stuff," focusing on generic AI applications that any competitor could replicate. The successful use cases, like Royal Bank of Canada automating equity research reports or Merck using AI for drug discovery (Teddy), are deeply intertwined with the specific data and complex analytical processes of those organizations. Similarly, 7-Eleven's automated marketing stack leverages granular customer segmentation and content generation tailored to their specific retail operations. These aren't just about applying an LLM; they are about engineering sophisticated AI solutions that understand and act upon unique organizational knowledge.

The Hidden Cost of Speed and The Power of Delayed Payoff

The allure of AI is its perceived speed and efficiency. However, the conversation highlights how chasing immediate gains can lead to significant downstream problems. Arvind Jain shares a personal anecdote about wanting an AI agent to define weekly priorities for everyone in the company, a seemingly simple task. Yet, he admits, "I still don't have it." This illustrates that even straightforward applications of AI require substantial engineering effort and time, and the "magic" of AI doesn't bypass the complexities of organizational design and human behavior. The immediate payoff is often elusive, and the effort involved can be substantial, creating a disincentive for many.

This difficulty and delayed payoff are precisely where competitive advantage can be forged. Ghodsi and Jain are building companies that are not just about deploying AI models but about integrating them deeply into workflows and data infrastructure. Glean, for example, aims to automate the massive coordination overhead within organizations--the endless meetings, emails, and documentation that consume employee time. This is not a quick fix; it requires building a comprehensive understanding of an organization's data and communication patterns.

"The pattern repeats everywhere Chen looked: distributed architectures create more work than teams expect. And it's not linear--every new service makes every other service harder to understand. Debugging that worked fine in a monolith now requires tracing requests across seven services, each with its own logs, metrics, and failure modes."

-- (Paraphrased analysis of complexity in distributed systems, common theme in enterprise AI implementation)

The contrast with Robotic Process Automation (RPA) is instructive. While RPA offered automation, it was brittle, rule-based, and lacked learning capabilities. AI, by contrast, is a "learning agentic system." This fundamental difference means AI can generalize, understand patterns, and improve over time. However, as Jain notes, the current paradigm of freezing models after training presents challenges. Truly agentic systems that can learn and adapt continuously are still a frontier. The companies that can navigate this complexity, invest in the engineering art of making AI work within specific enterprise contexts, and patiently build these capabilities will reap the rewards. This requires a long-term vision that prioritizes durable advantage over short-term wins, a mindset that is often at odds with the rapid pace of technological change and the pressure for immediate ROI.

Navigating the AI Budgeting Maze and The Future of Value Accrual

For Chief Information Officers (CIOs) planning AI budgets, the advice from Ghodsi and Jain is pragmatic: experiment, but do so strategically. The market is nascent, with numerous players emerging, making it difficult to identify long-term winners. Their recommendation is to opt for shorter-term contracts and products that are easy to test and demonstrate value quickly. This "crawl, walk, run" approach allows organizations to learn and adapt without committing vast resources to unproven solutions.

The conversation also touches upon the immense capital expenditure in AI, particularly on the semiconductor side, and the question of how this investment will translate into revenue. Ghodsi and Jain present three camps: the "super intelligence quest" camp, which is consuming significant capital in pursuit of AGI; the "researcher" camp, focused on fundamental breakthroughs but largely unfunded; and their own camp, focused on making AI valuable now within enterprises. They argue that while the quest for superintelligence may eventually yield immense economic value, current enterprise AI is about leveraging existing AGI capabilities to solve real-world problems.

"We have AGI. We really have it. It's like anyone who says we need to get to AGI -- that's like it's a false premise to start with. We already have AGI."

-- Ali Ghodsi

This leads to the critical question of where value will accrue in the AI ecosystem. While models are becoming commodities, Ghodsi and Jain emphasize that the true value will reside in the data layer, the intelligence layer (which leverages proprietary data and workflows), and, most significantly, the application layer. They predict that while companies like Databricks and Glean will be valuable platforms, the ultimate beneficiaries will be the novel applications that emerge, much like how Facebook, Airbnb, and Uber became dominant in the internet era, rather than infrastructure providers. The key will be applications that offer proactive, personalized AI assistance, moving beyond current user interfaces to more natural interactions like speech.

The future of traditional software, often dismissed as "crud apps," is also re-evaluated. Ghodsi and Jain argue that these are not merely databases but complex ecosystems of workflows and applications. While AI will undoubtedly transform how these are built and interacted with, the end-to-end stack of software is likely to remain crucial. The challenge will be in how data is entered and utilized, with potential disruptions coming from platforms that can seamlessly capture and process conversational data, like Zoom, integrated with enterprise knowledge systems like Glean. This suggests a future where AI is not just a tool but a pervasive collaborator, augmenting human capabilities and driving efficiency, but only for those organizations that strategically invest in data, workflow integration, and novel applications.

Key Action Items:

• Prioritize Data Strategy: Before investing heavily in AI models, ensure your organization's proprietary data is clean, accessible, and well-understood. This is the foundation for building defensible AI advantage.
• Experiment with Commodity Models Selectively: Leverage readily available LLMs for initial experimentation, but focus on use cases that integrate with unique company data or workflows. Avoid building solely on generic LLM capabilities.
• Focus on Workflow Integration: Identify complex internal processes that are ripe for AI-driven automation and augmentation. The value lies in making these processes more efficient and effective, not just in applying a new technology.
• Embrace Delayed Payoffs: Invest in AI initiatives that may not show immediate returns but promise significant long-term competitive advantage. This requires patience and a willingness to endure initial discomfort or lack of visible progress.
• Adopt an Agentic Mindset: Train employees to view AI as a collaborator. Encourage the use of AI tools for tasks ranging from information retrieval to content generation, focusing on improving output quality rather than just saving time.
• Adopt Shorter Contract Cycles for New Vendors: Given the rapid evolution of the AI market, opt for flexible contracts when evaluating new AI tools and platforms to allow for easier pivots.
• Develop AI Literacy Across the Organization: Invest in training and education to ensure employees at all levels understand how to effectively use AI tools and contribute to AI-driven initiatives. This is crucial for bridging the gap between top-performing and average users.