Transitioning SRE Teams to Managed Specialized AI Agent Architectures

The future of Site Reliability Engineering (SRE) is not about replacing human judgment with AI. It is about moving the human role from firefighter to orchestrator. Asaf Savich, AI Engineering Group Manager at Komodor, explains that the main bottleneck in reliability work is not a lack of data, but a surplus of noise without context. By moving away from monolithic AI models toward specialized, scenario-tested sub-agents, organizations can reduce token costs and hallucinations while improving accuracy. Teams that stop treating AI as a black box and start treating it as a managed workforce of specialized agents will reclaim the time currently lost to 2:00 AM incident response, allowing them to focus on infrastructure health and cost optimization.

The Hidden Cost of Context Dumping

SRE teams often assume that more data leads to better AI performance. Savich notes that feeding an LLM every available log, metric, and trace creates an operational nightmare. This approach ignores token limits and dilutes the focus of the model.

The system responds to this context dumping with increased latency and higher rates of hallucination. Savich argues that the solution is not a larger context window, but a more rigorous delegation strategy. By using specialized sub-agents, each optimized for narrow tasks like log summarization or database connection analysis, the primary agent remains efficient.

If I would take like a generic log summarization agent it does not have the right context in order what should they summarize, what should they look at et cetera, et cetera. So now we have the sub-agent... This makes our main agent much more efficient, much faster, less tokens and less chances to hallucinations.

-- Asaf Savich

Evaluation as a Competitive Moat

The most overlooked dynamic in AI engineering is the need for a static, trusted evaluation layer. While production incidents are dynamic, effective AI development requires static scenarios: simulated, noisy environments where the same failure can be reproduced thousands of times.

This creates a pyramid of velocity. Without a trusted judge to define what a correct investigation looks like, teams are flying blind. Savich emphasizes that accuracy is the goal, and achieving it requires thousands of iterations of A/B testing between models. Teams that invest in building these internal evaluation frameworks gain an advantage over those relying on off-the-shelf performance, which Savich notes is not good enough for production stability.

The Shift from Building to Managing

There is a recurring tension between building custom orchestrators and relying on established platforms. Savich observes that teams often try to build everything in-house, only to realize that garage-built tools lack the support and depth required for complex cloud-native environments.

The real shift, according to Savich, is in the role of the SRE. As AI agents move from detection to remediation, the human engineer becomes a manager of agents. This is a high-leverage transition: instead of manually parsing logs, the SRE reviews remediation plans and approves workflows.

I think AI service will manage other people, the agents and their own agents. And the reviewers, the approvers of walkflows and let like the agents do the role of putting much less tension on their shoulders of their stories that exists today.

-- Asaf Savich

This shift creates a feedback loop: by offloading firefighting to managed agents, SREs gain the bandwidth to focus on preventive strategy, which stops incidents before they occur.

Key Action Items

• Implement a Judge Layer (Next 30 days): Stop relying on intuition. Create a static test suite of past incidents where the correct root cause is known. Use this to evaluate every AI model update before it touches production.
• Adopt a Sub-Agent Architecture (Next 30-60 days): Break your primary AI agent into specialized roles. If your agent is failing at database issues, build a specific sub-agent for that domain rather than trying to prompt-engineer the generalist model into better performance.
• Shift from Full Automation to Human-in-the-Loop (Immediate): Do not aim for fully autonomous remediation yet. Focus on generating remediation plans that an engineer can approve. This builds trust and creates a safety net that prevents catastrophic automated mistakes.
• Build a Memory Layer (Next 3-6 months): Ensure your AI has access to your organization's past post-mortems. Connecting the agent to your internal knowledge base allows it to learn from institutional history, not just current logs.
• Prioritize Preventative Analysis (12-18 months): Once your investigation agents are stable, pivot them to analyze infrastructure for common, repetitive issues like connection leaks or Helm configuration errors to stop problems before they trigger an incident.