GPU-Accelerated Storage Enables In-Place AI Data Preparation

Key Quotes

"but it's still challenging for enterprises to put agents into production so it sounds like the technology is there or you know there enough because it's always advancing right what are the challenges then um that enterprises are facing to actually deploy and gain adoption of agents that's a great question so almost every enterprise and every ceo and cio cto they are increasing their spending on ai they want to deploy ai's to production but it's difficult to move from a poc state to production so the enterprise adoption is happening but it's still challenging there are still challenges can you dive into what some of the challenges are yes so many of the challenges revolve around access to data"

Jacob Lieberman explains that while AI agent technology has advanced significantly, enterprises face challenges in moving from proof-of-concept (POC) stages to full production deployment. He identifies that a primary hurdle is ensuring secure and efficient access to accurate, up-to-date data, which is fundamental for any AI application, including agents.

"regardless of how you're using ai whether it's an agent whether you're training a model whether you're fine tuning a model or even if you're retrieving data to feed it to an llm's context all ai still relies on secure access to accurate recent data great so you know you mentioned the human factor and that's uh separate episode separate podcast in itself right because that i mean for all of us change you know and this is we have to bring lawyers and hr folks for that one right exactly when it comes to the the systems and you know the legacy infrastructure and that kind of thing is it a situation where like you kind of have to burn it down and start from scratch or can you kind of you know update and transform existing data to be ai ready"

Lieberman emphasizes that all AI applications, from agents to model training, are fundamentally dependent on secure access to current and correct data. He notes that the challenges in enterprise AI adoption extend beyond the technology itself, touching upon human factors and the integration with existing legacy infrastructure, posing the question of whether a complete overhaul or an update of existing systems is necessary.

"there is this term that's emerging called ai ready data ai ready data has gone through a number of transformations to make it efficiently and securely useful by ai systems like retrieval augmented generation or data flywheels or ai agents and making enterprise data ai ready is challenging it's challenging because the vast majority of enterprise data is unstructured it has no structure to it it's it's things like powerpoint presentations and pdfs and audio files and videos and all kinds of things that you can't just shove into a database and query with a structured query language"

Jacob Lieberman introduces the concept of "AI-ready data," which has been transformed for efficient and secure use by AI systems. He highlights that making enterprise data AI-ready is difficult, primarily because most enterprise data is unstructured, existing in formats like presentations, PDFs, audio, and video, which cannot be easily queried with traditional structured query languages.

"the most typical use case for ai inference is retrieval augmented generation and to make your unstructured data ready for rag you have to collect all the data extract all the text the semantic knowledge within the data chunk it up into homogeneous sizes enrich it with metadata then embed it which means transform it into numeric representations that can be efficiently stored and searched and at the very end you index it into a vector database for retrieval and you're doing this at enterprise scale"

Lieberman details the process for preparing unstructured data for Retrieval Augmented Generation (RAG), a common AI inference use case. This involves collecting data, extracting semantic knowledge, chunking it, enriching it with metadata, embedding it into numeric representations, and finally indexing it into a vector database for retrieval, all performed at an enterprise scale.

"the big challenge to enterprises is from a governance standpoint they often don't know which data has changed and if you don't know which data has changed you have to reindex all of it so uh that reminds me you know when i'm at home and let's say i put a dish into the dishwasher and it's dirty and i don't know that the dishes have already been washed and then you know my wife tracy says to me hey which dish did you put in there and i can't remember so we end up having to rewash everything and that's what enterprises are doing right now to make their data ready they're rewashing all their dishes every time they have one dirty dish"

Jacob Lieberman uses an analogy of rewashing all dishes when unsure if they are already clean to explain a significant enterprise data governance challenge. He states that enterprises often lack visibility into which data has changed, forcing them to re-process or re-index all data to ensure it remains AI-ready, which is inefficient and analogous to rewashing a full load of dishes when only one is dirty.

"when you put your data into a pipeline to make it ai ready that generally involves copying the data you don't transform the source of truth documents you make a copy and you chunk that extract text embed that and every time you make a copy you're increasing the attack surface on your data and it becomes even more threatening if you have to move that data out of your storage because as soon as you're copying the data somewhere else it becomes disconnected from the source of truth so that means if the contents change like we just said we call that data drift the content changes might not be reflected in your ai representations but what's even worse is what if the permissions change what if noah should no longer have access to this document right his permissions are removed but he can still access all the copies and we're seeing that in a typical enterprise as chatbots and agents proliferate we're seeing multiple copies of the same data set 7 13 copies appearing all around the data center disconnected from the source of truth"

Lieberman points out a critical security issue in traditional AI data pipelines: data copying. He explains that creating copies of data to make it AI-ready increases the attack surface and creates risks, especially if permissions change. He notes that data drift and outdated permissions on these copies are common problems as AI applications like chatbots proliferate, leading to multiple, disconnected copies of data across the data center.