Moving from Retrieval to Curation: Why RAG Is No Longer Enough

In this conversation, Jörg Schad, VP of Engineering at Pinecone, outlines the shift from real-time retrieval to precomputed knowledge engines. While Retrieval-Augmented Generation (RAG) solved the initial problem of grounding models, it created a hidden, compounding cost: the ETL-on-the-fly tax. By treating context as a versioned asset, similar to a materialized view in database systems, organizations can move from probabilistic, inconsistent results to deterministic, audit-ready AI systems. This transition is a fundamental change in how we treat data as a product for agentic consumers. For engineering leaders and architects, mastering this transition provides a durable competitive advantage: the ability to build AI systems that are reliable, governable, and performant at scale.

The Hidden Cost of On-the-Fly Retrieval

The conventional RAG pattern, where an agent searches a vector database during every query, feels productive in the short term. It solves the immediate problem of getting data into the model. However, Schad argues that this approach creates a downstream operational nightmare. Because the agent performs retrieval, curation, and formatting on every query, the system becomes inherently probabilistic and difficult to debug.

If I am doing that at each individual query, I am doing it over and over again. And LLMs or agents are just probabilistic systems. They are choosing something on the fly. And so if I actually care about reproducible results, if I am asking my data a question like what has been the revenue last year? I really want a consistent answer over time.

-- Jörg Schad

When you treat context as a first-class citizen, or a precomputed asset, you gain the ability to apply permissions, lineage tracking, and versioning. This creates a separation of concerns: the curation logic is decoupled from the agent query. Over time, this reduces the sub-loop complexity that plagues current agentic architectures, where the agent spends its expensive token budget on repeated, redundant retrieval work.

Where Immediate Pain Creates Lasting Moats

Most teams optimize for the immediate win of getting a prototype working. Schad suggests that the real advantage lies in the work of lineage and governance. By building a system where context artifacts carry their own metadata, such as freshness, source lineage, and semantic definitions, you create a moat of reliability.

When your competitors are debugging why their agents gave three different answers to the same revenue question, your system provides an audit trail showing exactly which version of the curated context was used. This is the difference between a toy agent and an enterprise-grade knowledge engine.

I remember spending a lot of nights trying to rebuild this lineage tracing, because we had to debug production issues. I basically had to justify why was this here in the future? If you are going back to feature stores, why is this information ending up in features?

-- Jörg Schad

This requires patience that most teams lack. It demands that you treat your AI context as a formal data product, with the same rigor you would apply to a production database schema.

The System Responds: Evolving the Knowledge Engine

Systems thinking reveals that as you improve your context curation, your agents will adapt. Initially, you might provide a simple vector index. But as you observe the questions your agents struggle with, you can iteratively improve the shape of your context. Schad notes that you can use idle compute time to perform background curation, using graph machine learning to identify new connections or Bayesian updates to adjust confidence intervals on data edges.

This shifts the burden of intelligence from query time, where it is expensive and slow, to compilation time, where it is efficient and scalable. The system responds by becoming faster and more reliable, creating a feedback loop where the more you curate, the more capable your agents become at planning and executing complex tasks without needing to re-learn the data on every turn.

Key Action Items

• Move from on-the-fly to materialized: Over the next quarter, evaluate your RAG pipelines. Identify which queries are repeated and begin materializing those context sets into versioned artifacts rather than re-fetching them.
• Implement lineage tracking: Begin tagging context artifacts with metadata, such as timestamp, source, and curation version. This pays off in 6 to 12 months when you need to perform root-cause analysis on an incorrect model output.
• Define your semantic layer: Stop relying on the model to guess definitions. Create a shared semantic layer for key terms, such as fiscal year or revenue, to ensure consistency across different agentic tools.
• Adopt No-QL or structured interfaces: Move away from raw, unstructured retrieval. Invest in a structured query interface that allows agents to request data with specific schema constraints. This creates a more deterministic interaction layer.
• Iterate on curation, not just prompts: Instead of spending all your time tuning system prompts, spend the next 12 to 18 months building the curation program that generates your context. This is the long-term investment that creates separation from competitors.
• Build for confidence metadata: Start experimenting with confidence scores on your data edges. If an agent knows a fact is only 50 percent certain, it can plan a different, more cautious path, a capability that simple RAG systems lack.