Adapting Identity Protocols for Non-Deterministic AI Agent Security

Key Quotes

"the way i think about it and this came up in a conversation the other day is it's really like a workflow or a set of pieces of software that can like accomplish workflows which we've had for decades right workflow software's been around for a long time the difference is that the workflow used to be defined in code or in stack configuration and now it's more in natural language and so the idea you can actually kind of give an agent a task in relatively natural language and have it not to overuse the word grope but grope towards accomplishing it which is what they do is to some extent a game changer"

Dan Moore explains that AI agents represent a shift from traditional workflow software, where processes were defined in code or configuration, to a more natural language-driven approach. This change is significant because it allows tasks to be initiated and guided using everyday language, fundamentally altering how workflows are created and executed.

"the way i think about it is that human beings are slow and non deterministic and software as written you know five years ago 10 years ago even the vast majority of software right now is fast and relatively deterministic right charity majors might have some issues about that how deterministic it is but it's pretty deterministic and easy to reason about whereas agents kind of fall in this middle ground and so they are fast and non deterministic"

Dan Moore highlights a key distinction between human behavior, traditional software, and AI agents regarding determinism and speed. He notes that humans are slow and non-deterministic, while most existing software is fast and deterministic. AI agents, however, occupy a middle ground, being fast yet also non-deterministic, which introduces new complexities.

"so there's this great thing called the lethal trifecta which simon willison who's written a ton about ai and llms in general talks about and the idea is that agents have access to private data which again software does too it's exposure on trusted content which again software does too and then there's the ability to externally communicate and so the issue is that because agents are non deterministic and because they read that untrusted content they can then be instructed to access your private data and then send off an email and this is a real i mean i would consider this to be a new threat because because of the ability to follow like arbitrary instructions"

Dan Moore introduces Simon Willison's "lethal trifecta" concept to describe a new threat model posed by AI agents. This trifecta involves agents having access to private data, exposure to untrusted content, and the ability to communicate externally. Moore emphasizes that the combination of non-determinism and the capacity to follow arbitrary instructions makes it easier for agents to misuse private data, representing a novel security concern.

"non deterministic in my mind means that or sorry let's start with deterministic deterministic is you give the same inputs and you get the same outputs so math is a perfect example of a deterministic system gravity to some extent is too right whereas non deterministic means you give it the same inputs and you not necessarily get the same outputs whether human society conversations with your partner like these are all things that are non deterministic because they're affected by the current state of things and so if you've ever asked the same question of an llm you're going to get back i mean here's the thing like they're they're slightly different right or they could be kind of massively different depending on on the way that well i should be careful here because i'm definitely not an llm like creation expert so but at the end of the day it's it's not a system where you put in the same thing and you get the same thing out"

Dan Moore clarifies the concept of "non-deterministic" by contrasting it with "deterministic" systems. He explains that deterministic systems, like math or gravity, produce the same output for the same input. In contrast, non-deterministic systems, such as human conversations or LLM responses, can yield different outputs even with identical inputs, due to various influencing factors.

"i think actually that points to one of the solutions to this is that you start to have these kind of sub agents that you know may have access like i mean one way to deal with the that lethal trifecta i mentioned is that like you have different agents that have access to different sets of tools or different sets of data and if you have one that can read your email and another that can send your email it's going to be really hard for an attacker to be able to kind of get that email you know that private data sent off because you have to like basically now attack two different llms so kind of more separation of concerns which again this is not software it's not revolutionary right it's a lot of the same principles we're just having to apply it in a new context right separation of concerns is a great thing for security"

Dan Moore proposes a solution to the "lethal trifecta" by advocating for the use of "sub-agents" with distinct access privileges. He suggests that by segmenting access to tools and data among different agents, such as one agent for reading email and another for sending it, it becomes significantly harder for an attacker to exfiltrate private data. Moore emphasizes that this approach leverages the established security principle of separation of concerns, applying it to the new context of AI agents.

"i think that granular permissions are the way to go for sure like and if you you know i think that the two paths you can lay out are like api keys and oauth tokens right or access tokens and i feel like to some extent we talked about echoes of the past like this whole ai making like software like calling other software services feels a lot like the api landscape of like the 2010s and we've been through that and i'm not saying oauth is perfect i'm not saying bearer tokens are perfect but there's like a lot of infrastructure around it and it's kind of a well known protocol and so i think that oauth tokens properly constrained with scopes and probably fine grained access behind that rback our back pback we talked about that a little bit is probably the path forward"

Dan Moore asserts that granular permissions are essential for securing AI agents, likening the current landscape to the API environment of the 2010s. He suggests that OAuth tokens, when properly constrained with scopes and supported by fine-grained access controls like RBAC (Role-Based Access Control) or ABAC (Attribute-Based Access Control), represent the most viable path forward. Moore acknowledges that OAuth is not perfect but highlights its extensive infrastructure and widespread recognition as key advantages.