How Everpure plans to stop AI from starving without data

Imagine you are an AI Agent. You execute inside a $20-40 million-plus Nvidia SuperPOD system’s accelerator hardware, have a skill set and can call up other agents to do your bidding. An insurance company customer, using a PC browser or smart phone, asks a question: “Am I covered for weather-related damage under my current policy?” and it is handed off to an AI agent. What happens next? The agent itself runs on clusters of CPU + GPU/accelerator servers that host the large language model, any Retrieval-Augmented Generation (RAG) components, orchestration logic, and tools that look up the actual policy data. Those same servers (or tightly coupled backend systems) access the policy database/storage to answer the weather insurance coverage question, then return the response to the user’s device. Everything the agent does depends upon data, and it and all the other agents operating at the same time in the SuperPOD system, need to get that data off a storage system. A national or large regional-level insurance company will have petabytes, even exabytes, of data it stores so it can manage its insurance business. There needs to be a central index of this data’s structure, state, location, field names, types and semantics so that AI agents go to one place, a kind of data encyclopedia, to find out what information they need and where it is located. Agents are in a hurry, because GPU servers are hugely expensive and just a minute of idle time can cost $25 or more, a dollar every 2.4 seconds. The agent answering this insurance request needs data and a storage system needs to deliver it as fast as possible so that the GPU servers don’t wait and wait and wait some more for data. This requires the storage system to be able to handle the scale and types of the data involved, for it to be self-describing, and its software to be tightly integrated with Nvidia’s GPU hardware and software server architecture so that there isn’t a wasted millisecond while the GPUs are starved. A modern approach to data processing for AI Par Botes, Everpure’s VP of AI Infrastructure, discussed the difference between traditional enterprise critical data processing and how it’s done in the AI era: “The characteristics of performance, resilience, availability and governance don't fundamentally change, but access patterns are different. When searching for information, metadata can become richer than the data itself because it explains the semantics of data, and how it behaves as it becomes enriched.” Building an AI data architecture around standard dual-controller arrays or siloed direct-attached storage (DAS) quickly hits physical limits. What organizations need is an AI Data Platform, built using the NVIDIA AI Data Platform reference design. There are three distinct bottlenecks involved in this which Everpure’s architecture focuses on solving. First: Throughput starvation at scale: To keep thousands of GPUs saturated, a storage hardware platform needs to scale out past 4 or 5 TB/sec to 10 TB/sec or more of bandwidth. This rules out standard dual-controller arrays, even if equipped with SSDs instead of disk drives, or even cache-centric architectures. At the high-end enterprise level, an Everpure FlashBlade//S, or at the extremely large scale, a FlashBlade//EXA, provides a highly powerful data storage platform built for AI and HPC. With enterprise AI factories expected to scale out to, and perhaps beyond, 10,000 GPUs, there needs to be a solution which can handle this highly performant scale: FlashBlade//EXA scales up unstructured data storage technology to a whole new level for metadata controllers using NVMe fabric-connected flash for the data nodes. It’s Nvidia AI Data Platform-certified, delivering 400 to 450 million IOPS, 220 GB/sec bandwidth and 4.6 billion metadata operations/sec. Second: The KV cache prefill tax: A less obvious efficiency drain occurs inside a GPU’s High-Bandwidth Memory (HBM). Imagine a finance person’s AI model wants to read a business’ SEC 10-K report. It gets chunked, turned into tokens, then vectors, and loaded into the KV cache; the LLM pre-fill stage. When the model completes its run, the tokens are tossed away. But a second finance person could want their model to check the same 10-K but using a different GPU. So, the prefill is done again with, for example, 150,000 tokens recomputed. When this happens, the existing tokens need to be temporarily moved to an extended memory area instead of being evicted and lost, meaning GPU cycles have to be spent to recompute data that’s already been computed. Everpure Key Value Accelerator (KVA) offloads the cached token states directly to shared flash over NVIDIA GPUDirect Storage (GDS) via RDMA. When a subsequent query references that document, the pre-computed KV tensors stream straight into GPU memory, bypassing host CPU overhead and eliminating the compute penalty. Botes said: “It turns out that many, many patterns are far more reusable than people think. So, the more those patterns are reused, the more we can save compute time.” Physical proximity matters, Botes said: “Data speed drops dramatically as it moves further from the CPU or GPU: nanoseconds for local memory, microseconds for disk, and milliseconds for network transfers. Minimizing this distance is critical for performance. If I put that into scale, if memory access is a second, then disc access is an hour. The farther away you move, the more time everything takes and that doesn't work for efficient AI.” Third: Eliminating irrelevant data silos and maximizing model training: Data copying at scale is becoming impossible. An enterprise with data in different silos all over its estate is simply not going to move and copy that data into a single data warehouse or lake. Botes said: “You can't, the gravity is too expensive. [And] the second that data's changed at the source, that data lake is invalid.” Everpure Data Stream is this uniform mechanism, a data re-use platform, that ingests, curates and transforms data, vectorizing it, indexing it and serving it when requested. Data ingestion is automated and sources include the gamut of enterprise data: block, NFS file, S3 object, databases, data warehouses, data lakes, streaming data and enterprise applications, text documents, PDFs, images, and structured tables. It has built-in vector databases with NVidia’s NeMo Retriever enabling GPU-accelerated vector embedding generation, ensuring precise retrieval in RAG pipelines, as well as Nvidia NIM for deploying optimized inference. There are pipeline orchestration, GPU integration, Nvidia NIMs and NVAIE (Nvidia AI Enterprise) APIs plus ones for semantic search and LLM integration. Everpure Data Stream uses Nvidia’s RTX PRO 6000 Blackwell Server Edition GPU and libraries of code to synchronize data delivery across ConnectX-7 NICs with processor cycles: Nvidia Spark Rapids and cuVS. Everpure Data Stream ensures orchestration occurs at the storage level, with FlashBlade acting as a layer where metadata enrichment and relevance reranking transformations are executed in parallel, driving down end-to-end inferencing latency. Everpure Data Stream is a data factory and supply chain delivery system, finely tuned to operate in the Nvidia GPU server and software environment. It supports petabyte-scale RAG data sets, with independent scaling of capacity and performance to accommodate multiple GPU clusters without downtime. Botes said: “Without a RAG mechanism, a model is trained with knowledge from only one moment - by definition it’s always behind. It's at the point in time when it was trained. So what we do with the RAG is, we augment that model with knowledge that is from this very moment. That knowledge can only exist within a storage system. It never exists in a compute system. The more up to date we can make a model, then the more concise and better we can make the answers.” Everpure Data Stream is model-agnostic, Botes said: “Customers pick the models that best suit the data they’re interested in and configure the models.” Optimizing data placement and reuse For our imagined insurance AI agent, proximity between data and GPU is another important element to factor in. Placing data storage right next to the GPU server (as FlashBlade//EXA should be) avoids time-consuming data movement - adding value and providing optimal data placement. With this, there is a memory hierarchy to keep in mind. This kind of repeat token processing and the need for co-located GPU compute and storage is characteristic of AI models and agents operating in the finance analytics, healthcare clinical systems, legal corpora, real-time fraud detection, and cyber-security threat intelligence areas. The requirement for data primacy and data context The whole system downstream of GPUs has to be data-centric with a single logical data catalog or encyclopedia acting as the sole reference platform for AI agents to provide governance, look for and access the data they need. It pre-prepares data for AI use. Exabyte-scale data has enormous gravity and moving it is to be discouraged. But the metadata describing it, petabyte-scale metadata, has gravity too and needs to be stored close to the GPU servers. Metadata performance is key, Botes said: “FlashBlade is an incredibly fast, highly distributed database that can look up key-values and present them as storage protocols extremely quickly.” Botes said: “Hardware innovation is exciting and accelerating rapidly, bringing advancements like faster PCI Express bus speeds, high-bandwidth memory, and improved network interconnects. The only way for the technology to take advantage of the hardware innovation without reinventing infrastructure every single year is to disaggregate,” meaning separate compute from storage, hence the FlashBlade//EXA’s design. “For AI platforms requiring cutting-edge performance, it’s so important to capture these benefits immediately, and at the pace of GPU innovations. How effectively we can manage and deliver data is the most significant impact of this hardware innovation.” He explored this concept further: “Disaggregating compute, network, and storage allows us to scale them independently. This is crucial because innovation happens at different speeds for each component, preventing the need to overhaul the entire infrastructure every time one component advances.” This also allows organizations to upgrade hardware tiers independently through Everpure’s Evergreen subscription model, avoiding destructive forklift overhauls every time a new accelerator architecture arrives. A modern approach means a storage platform for AI agents has the basic IO speed to keep GPUs busy. It also has Everpure Data Stream software to ensure the right data IOs take place, and the data gets to the GPUs in the right way; data logistics as it were. Distributed data spread across an organization’s IT estate is discovered using Everpure Data Intelligence, following Everpure’s acquisition of 1touch. Botes said: “We are trying to make all storage layers self-describing, regardless of who makes them.” Everpure acquired 1touch to deepen data management capabilities and build a “data intelligence system. So we can probe it, describe it, and create indexes to understand it.” And: “We have to do transformation while accessing it.” The self-description add-on, as it were, is happening inside Everpure Data Stream. Isolated islands of truth Botes said: “When we classify, the key thing to think about is, anyone can hook up a system that can read all your data and make it into vector space and search it, kind of what OpenAI does. But the tricky part is different data needs different governance.” “When you classify, you also want to set up separate streams for separate governance rules, because not all data is the same. Every company has many different sources of truth: salesforce.com; an engineering system; the finance system, the HR system. They relate to each other, but they don't need the same governance, or access. There needs to be a uniform mechanism for searching and interacting with preserved governance.” Afterall, he said: “You don't want to have 55 little islands of isolated truths. We've been down that path before and it's really problematic. To date, the application has been the governor of truth. But AI doesn't work that way. We have to create ways to find data by reference, not by location or URL.” He believes that “We have moved to systems that make data self-describing, so that, while we can maintain that source of truth within the application, the data there can also be used by other things.” Because of self-description, he adds: “We're making it queryable through AI and interacting through AI without necessarily having to make a full copy of it. Our capabilities in data intelligence makes it easy to deploy for enterprise data.” Data centric architecture supporting AI workloads For the insurance AI agent mentioned, maximizing the data is the key to returning faster answers, making the most of any AI investment and delivering more ROI to businesses. As enterprise AI factories transition from pilot projects to scaled production, raw GPU count is fading as the primary metric of AI capability. The real benchmark is GPU utilization efficiency—and the battle to eliminate idle cycles is increasingly being fought at the storage layer. The Everpure Data Platform approach to AI has the scale and software intelligence to prevent GPUs waiting for data and avoid them wasting their precious and expensive processing cycles recomputing data they have already wrangled. Sponsored by Everpure