Prioritizing Attacker Utility Over AI-Generated Vulnerability Noise

Original Title: Exploit Development, Agentic Vulnerability Discovery, and Fake PoCs

The AI Vulnerability Arms Race: Why Speed is the New Moat

Spencer McIntyre, Director of Exploit Development at VulnCheck, explains the shift in cybersecurity as AI changes how software vulnerabilities are found and weaponized. We have reached a point where AI is excellent at finding theoretical flaws but often fails to turn those findings into functional, high-impact exploits. This creates a large volume of broken proof-of-concept (POC) code that distracts security teams. For business leaders and practitioners, the advantage comes from developing the operational maturity to tell the difference between theoretical risk and actual attacker utility. Those who ignore how systems and attackers adapt to these tools will be overwhelmed by the volume of AI-generated alerts.

The Noise Floor of AI-Generated Exploits

Conventional wisdom suggests that more automation leads to better defense. However, McIntyre points to a more complex reality. As AI tools lower the barrier to entry for vulnerability research, the volume of public proof-of-concept code has grown. The hidden consequence is a massive increase in broken or illegitimate POCs.

McIntyre notes that by 2026, over 50% of public POCs failed to meet professional standards for functional exploitation. This creates a drag on resources: security teams must triage a flood of AI-generated research that often lacks the attacker utility needed to actually compromise a system.

> "What we're seeing is that today your average POC that you find it's probably not what you're hoping it would be where it's actually going to truly leverage the vulnerability it says it does and prove some kind of attacker utility."
>
> -- Spencer McIntyre

The Limits of Automation and the Human Edge

While AI models are effective at identifying vulnerabilities in open-source web applications, such as logic flaws or expression language injections, they struggle with proprietary, closed-source software. These systems require reverse engineering and patch-diffing, tasks where AI currently lacks reliability.

This creates a competitive advantage for human-led teams. Because humans understand business impact and can refine exploits for specific, high-value targets, they remain the deciding factor in high-stakes security. The current system rewards those who use AI to handle the first line of triage, identifying the flaw and the trigger, while reserving human capital for the complex task of weaponization.

How Systems Route Around Your Guardrails

The conversation highlights a friction point: the guardrail dilemma. As proprietary AI models become more restrictive to prevent malicious use, legitimate security researchers are hampered. This encourages researchers to move toward local, open-weight models like GLM or Qwen.

This shift has a second-order consequence: it decentralizes the security research landscape. When researchers host their own models to bypass restrictive guardrails, they lose the benefit of centralized, vetted intelligence. The system forces teams to choose between the safety of proprietary, guarded models and the raw, unmonitored utility of local, open-source alternatives.

> "The trends in the guard rails definitely seem to have gotten more strict. If you talk to anyone in offensive security research, basically the answer is if you're using one of the American proprietary models, it's to downgrade it, that's the thing."
>
> -- Spencer McIntyre

The 18-Month Payoff: Why Speed Matters

McIntyre argues that the goal of defensive AI is to compress the window of vulnerability, which is the time between an exploit's discovery and the deployment of a patch. By automating the analysis of new vulnerabilities, defensive teams can neutralize the advantage nation-state actors have held. The payoff requires building internal frameworks, such as VulnCheck's Go Exploit, that normalize how teams interact with vulnerabilities. This investment pays off over months, allowing for rapid, repeatable response when the next Log4J-style event occurs.


Key Action Items

  • Audit your external-facing dependencies (Immediate): Identify how many applications in your stack use languages like PHP, which are frequent targets for widespread vulnerabilities.
  • Establish a triage framework (Next Quarter): Move away from relying solely on CVSS scores. Develop internal criteria for attacker utility to filter out noise from the surge of AI-generated POCs.
  • Invest in internal orchestration (3-6 Months): Stop building exploits from scratch. Implement or adopt a normalized exploitation framework that allows your team to interact with vulnerabilities consistently, regardless of the source.
  • Evaluate your AI toolchain (6-12 Months): Determine if your team's research is being throttled by proprietary guardrails. If so, assess the feasibility of hosting local models for internal research, keeping in mind the trade-off in data sovereignty.
  • Focus on high-value targets (12-18 Months): Because AI is currently better at exploiting open-source logic flaws than closed-source proprietary systems, prioritize your human security resources for the latter.

---
Handpicked links, AI-assisted summaries. Human judgment, machine efficiency.
This content is a personally curated review and synopsis derived from the original podcast episode.