Proactive AI Governance Essential to Mitigate Sprawl Risks

Key Quotes

"Because, yeah, some companies have approved AI tools, others don't, and sometimes you have entire departments using unsanctioned AI, kind of this shadow AI, that's casting a huge shadow over, "Are you even getting a return?" It is extremely difficult for decision-makers in organizations to not just know, "Hey, what's the right platform to use? Maybe take advantage of the best models regardless of the provider," but also how to do it in a secure way that can manage the chaos and actually lead to a positive ROI."

Jordan Wilson highlights the pervasive issue of "shadow AI," where departments or individuals use unapproved AI tools, creating a lack of visibility and making it difficult for decision-makers to track return on investment and ensure secure implementation. This situation complicates the landscape for organizations trying to leverage AI effectively and securely.

"Well, all of that sprawl has created the spaghetti mess that we're in now, where you've got all these different tools, each with their own management needs that may or may not be met, and that's huge security risks. You've got agents that are being created by employees, often being given employees' individual permissions, which are very broad versus maybe the agent was supposed to go do this one thing, but instead now it's been given all these capabilities to do a lot of other things that can create risk."

Kevin Kiley describes the resulting "spaghetti mess" from AI sprawl, where numerous tools require individual management, leading to significant security risks. He specifically points out the danger of employee-created AI agents being granted overly broad permissions, enabling them to perform actions beyond their intended scope and introduce potential vulnerabilities.

"It is often the well-intentioned employee that I mentioned to you, who, you know, they're really trying to move up, they've got a bandage to go build their piece of the business and drop some new product launch or whatever it might be. And they are not trying to do anything that would undermine security, but they're, of course, given the agent permissions, or maybe it's uploading sensitive data that in their context doesn't look that dangerous, but if a bad actor were to get to it, or your competition were to get to it, or someone else that could take advantage, that's where the risks really start to get introduced."

Kevin Kiley emphasizes that the primary concern with shadow AI often stems from well-intentioned employees seeking to increase productivity. He explains that these employees may inadvertently introduce risks by granting excessive permissions to AI agents or uploading sensitive data, which, while seemingly innocuous in their context, could be exploited by malicious actors or competitors.

"So back to your question, again, being able to provide almost like a model garden for your employees is an important part of this, where we can serve up a list of models that we consider to be trusted, or perhaps we'd already done the procurement work behind the scenes, so you don't have to worry about putting in a garden and, you know, all the different spend that's kind of gone out of control. Let's have a central view into where we're consuming, set up budgets, read and thresholds for notifications."

Kevin Kiley suggests that providing a curated "model garden" for employees is crucial for managing AI adoption. This approach involves offering a selection of trusted AI models, potentially with procurement already handled, and establishing a central platform for monitoring AI consumption, setting budgets, and configuring alerts to maintain control over spending and usage.

"Well, the first thing I would point to is again, that there's this arms race of innovation happening right now, where month to month, sometimes week to week, you're seeing a new leader take that top inch mark for performance accuracy. We just all continuing to be blown away by how quickly this space is evolving. And it's performance, it's the fact that they may be shedding that old model that you got hooked on, or maybe the cost implications, you know, if you started on ChatGPT 4.0 last year and somehow try to stay on that, very powerful and really again, was so exciting when it first came out, but it was also quite expensive."

Kevin Kiley highlights the rapid pace of innovation in AI, where new models quickly surpass previous leaders in performance. He points out that staying agile is essential because models can become outdated, and costs can fluctuate significantly, as seen with the evolution from earlier versions of ChatGPT to newer, more cost-effective iterations.

"Well, the big problem that has been well documented in a lot of different studies, and most recently there was the MIT Gen AI Divides, I think it's called the State of AI in Business report that was really kind of an earthquake across the industry, and I suspect a lot of your audience has had a chance to see this or at least read parts of it, where they saw that 95% of these pilots never get to production. And that's staggering when you think about all the money that's being spent and the good intention behind all the tools that are being purchased, implemented, consultants, you know, it's a lot of ways, unfortunately, that thus far has just not been able to deliver real value to the business."

Kevin Kiley references a significant finding from the MIT State of AI in Business report, indicating that a staggering 95% of AI pilots fail to reach production. He notes that this represents a substantial amount of investment and effort that has not yet translated into tangible business value, underscoring a widespread challenge in AI implementation.