Orchestrating Enterprise AI Agents: Beyond Individual Capability

In this crucial conversation from The Stack Overflow Podcast, Intuit's Chase Roossin and Steven Kulesza illuminate the intricate challenges of orchestrating multiple AI agents at enterprise scale. Beyond the surface-level excitement of generative AI, they reveal the non-obvious implications of building truly integrated AI systems. The core thesis is that successful multi-agent AI isn't just about individual agent capability, but about robust orchestration, rigorous evaluation, and a deep understanding of system dynamics. This discussion is essential for engineering leaders and architects grappling with the practical realities of deploying AI, offering a roadmap to navigate the complexities of integration, cost management, and the critical human element in AI evaluation. Those who grasp these principles will gain a significant advantage in building scalable, reliable, and customer-centric AI experiences.

The Unseen Architecture: Orchestrating Agents Beyond the Hype

The current wave of AI, particularly with the rise of generative models and agentic systems, often conjures images of individual AI entities performing complex tasks with near-magical autonomy. However, as Chase Roossin and Steven Kulesza from Intuit reveal, the real frontier lies not in the agents themselves, but in the sophisticated systems required to make them work together harmoniously at an enterprise scale. The conversation underscores a fundamental truth: building a single, brilliant AI agent is one challenge; orchestrating a symphony of agents, each with its own capabilities and limitations, to deliver a cohesive customer experience is an entirely different, and far more complex, endeavor.

The journey at Intuit, as described by Roossin and Kulesza, began with individual agents deployed in bespoke parts of their product. The evolution was driven by a customer-centric imperative: customers were spending significant time navigating disparate parts of Intuit's offerings. The vision became a singular, magical experience where agents collaborate to serve the customer, rather than requiring the customer to interact with each agent independently. This shift from isolated functionality to integrated intelligence is where the true complexity emerges. Steven Kulesza frames this as akin to building an organization: "You have all this distributed power across the platform, right? And like, how do we have all these things coordinate and work together? So that kind of drove like the whole thought process of like, how do we, yeah, go back to your main question of like orchestrating these multiple agents?" This analogy highlights that the problem is not just technical, but organizational and systemic.

The parallels to microservices architecture are evident, but the speakers emphasize unique challenges introduced by AI agents. While microservices deal with distributed systems, agents introduce a layer of non-determinism and emergent behavior. Kulesza points out that unlike distinct microservices, agents can appear similar on the surface, leading to difficult decisions about which agent to invoke. This ambiguity is compounded when dealing with complex financial data, where precision is paramount. The speakers highlight that AI, while excellent at reasoning and planning, often struggles with the mathematical exactitude required for financial applications. This tension between the generality of LLMs and the precision demanded by financial data necessitates a robust architecture.

"The core tension there is kind of knowing the strengths and the weaknesses of these models and what they're good at and leveraging what they're good at, you know, using determinism when we need to. So yeah, that core tension of generality and precision, especially with financial data, and, you know, that demands correctness on amounts, dates, entity names, while the agent layer needs to handle novel queries without a code deploy."

This necessitates a "skills and tools" approach. Instead of monolithic agents, capabilities are broken down into discrete, deterministic tools. A central planner then orchestrates these tools, allowing for complex, cross-domain queries that a single agent could never handle. Roossin explains the necessity of this shift: "customers don't just ask like a question that should go to one agent or this agent or that agent, right? Very commonly, they're getting cross-domain questions. For instance, like, you know, what would happen if I gave all of my employees a 5% pay bump, and my margins next month are going to go down? There's no one agent that would solve that, right?" This architectural evolution from a multi-sub-agent model to a flatter, skills-and-tools-based system is a direct response to the cascading complexity of real-world customer needs, demonstrating how system design must adapt to emergent user behaviors.

The Unseen Cost: Evaluation and Determinism at Scale

Perhaps the most significant hidden consequence revealed in the conversation is the immense challenge and importance of AI evaluation. Roossin and Kulesza emphasize that this is not merely a technical hurdle but a critical component for ensuring customer trust, especially in a sensitive domain like finance. The idea of an AI evaluating another AI's work is inherently complex. Intuit employs a multi-layered evaluation strategy: offline evaluations using LLM judges and golden datasets, online evaluations based on customer feedback, and human evaluations by domain experts. This layered approach aims to mitigate the limitations of any single method.

The problem of scaling human evaluation is acknowledged, leading to a nuanced approach where human oversight is strategically applied based on the complexity of the query and the need for customer comfort. This highlights a critical trade-off: immediate scalability versus long-term accuracy and trust. The decision to invest heavily in evaluations, even when it might slow down initial deployment, is a prime example of prioritizing long-term advantage over short-term gains.

"It's just this feedback loop at the end of the day, and we keep tuning those LLM judges, keep tuning our agents, keep tuning the prompts that are determining the intent detection there."

This continuous tuning and feedback loop is essential for maintaining the integrity of the system. Without it, the non-deterministic nature of LLMs could lead to unpredictable and potentially harmful outputs, especially when dealing with financial calculations. The commitment to evaluation-driven development, as Kulesza states, "evaluation first, person foremost," is what allows Intuit to build confidence in their AI systems and manage the inherent risks. This rigorous evaluation process, while resource-intensive, builds a durable system that can withstand scrutiny and deliver reliable results, creating a competitive moat for those willing to invest the effort.

The Latency Paradox: Managing AI's Operational Demands

Beyond the logic and evaluation, the conversation delves into the operational realities of running large-scale AI systems. Steven Kulesza articulates the unique challenges posed by LLMs compared to traditional cloud services. While scaling traditional services might involve simply adding more nodes, LLMs present different problems: capacity constraints, variable latency (sometimes 45 seconds or more for a single response), and the associated cost of token usage. This introduces a "latency paradox"--the desire for sophisticated AI interactions often clashes with the infrastructure's ability to deliver them quickly and affordably.

The speakers discuss the need for sophisticated fallback strategies, where the failure or unavailability of a primary model doesn't cripple the system but gracefully transitions to an alternative, equally evaluated, model. This requires a shift in the software development lifecycle, demanding greater cognizance of AI-specific operational concerns. Furthermore, managing token spend--both for inference and evaluation--adds another layer of complexity to cost management. Intuit's approach prioritizes customer needs first, then dials in cost constraints, a strategy that acknowledges that building a valuable AI experience is the primary objective, with cost optimization as a subsequent, but crucial, refinement.

"Capacity constraints were something of the on-prem days, you know, back when we were just running normal services on AWS. It's like, okay, dial up your HPA or whatever and, and you get some more nodes and you're, you're fine. You don't have that luxury. It's getting better, of course, right? But like, you know, capacity constraints with LLMs are always a struggle. And then these latency patterns are so different compared to what we're used to in, you know, modern technology."

This focus on operational excellence, robust evaluation, and careful architectural design--particularly the move to a skills-and-tools paradigm--is what allows Intuit to navigate the complexities of multi-agent AI. It’s a testament to the idea that building AI at scale requires a holistic approach, integrating technical prowess with a deep understanding of customer needs and operational realities. The advantage lies not just in the AI models themselves, but in the resilient, adaptable, and meticulously evaluated systems that support them.

Key Action Items

• Immediate Actions (Next 1-3 Months):

  • Audit existing agent capabilities: Identify discrete, repeatable tasks that can be refactored into "tools" or "skills" for a more modular architecture.
  • Establish a tiered evaluation framework: Implement a combination of automated (LLM judges, golden datasets) and human evaluations for new and existing AI components.
  • Map customer journeys: Analyze how customers currently interact with disparate AI features and identify opportunities for integrated, cross-domain experiences.
  • Track token spend rigorously: Implement enhanced observability for token usage across all AI interactions and evaluations to identify cost optimization opportunities.

• Longer-Term Investments (6-18+ Months):

  • Develop a central AI planner: Architect a system where a single, intelligent planner can orchestrate multiple tools and skills, enabling complex, cross-domain query resolution. This requires significant architectural shifts.
  • Invest in human-in-the-loop evaluation scaling: Explore strategies to scale human evaluation capacity or develop more sophisticated AI judges that can reliably approximate human judgment for common tasks.
  • Build robust fallback and retry mechanisms: Design for AI model unreliability and latency by implementing evaluated fallback models and graceful degradation strategies. This is crucial for uptime and customer experience. (Discomfort now for advantage later: Implementing these fallbacks is complex and time-consuming, but essential for reliability.)
  • Focus on "done-for-you" experiences: Shift development efforts towards AI systems that proactively complete tasks for the customer, reducing their effort and interaction time. This is the ultimate payoff for complex orchestration. (This requires significant upfront investment but promises substantial long-term competitive advantage.)