Here’s why 100TB+ SSDs will play a big role in ultra-large language models in the near future

Kioxia unveils new project called AiSAQ, which wants to replace RAM with SSDs for AI computing
Larger (read: 100TB+) SSDs could improve RAG at a lower cost than just using memory
No timeline has been given, but expect Kioxia’s rivals to offer similar technology

Large language models often generate plausible but factually incorrect output – in other words, they make things up. These “hallucinations” can damage the reliability of information-critical tasks such as medical diagnosis, legal analysis, financial reporting and scientific research.

Retrieval-Augmented Generation (RAG) addresses this problem by integrating external data sources, giving LLMs access to real-time information during generation, reducing errors and, by grounding output in current data, improving contextual accuracy. Implementing RAG efficiently requires significant memory and storage resources, and this is especially true for large-scale vector data and indexes. Traditionally, this data has been stored in DRAM, which, while fast, is both expensive and limited in capacity.

To address these challenges, ServeTheHome reports that at this year’s CES, Japanese memory giant Kioxia introduced AiSAQ — All-in-Storage Approximate Nearest Neighbor Search (ANNS) with product quantization — that uses high-capacity SSDs to store vector data and indexes. Kioxia claims that AiSAQ significantly reduces DRAM usage compared to DiskANN, offering a more cost-effective and scalable approach to supporting large AI models.

More accessible and cost effective

(Image credit: Kioxia)

Switching to SSD-based storage allows handling larger data sets without the high costs associated with extensive DRAM usage.

While accessing data from SSDs may introduce a small latency compared to DRAM, the trade-off includes lower system cost and improved scalability, which can support better model performance and accuracy, as larger data sets provide a richer basis for learning and inference.

By using high-capacity SSDs, AiSAQ meets the storage requirements of RAG while contributing to the broader goal of making advanced AI technologies more accessible and cost-effective. Kioxia hasn’t revealed when it plans to bring AiSAQ to market, but it’s a safe bet that rivals like Micron and SK Hynix will have something similar on the way.

ServeTheHome concludes: “Everything is artificial intelligence these days, and Kioxia is also pushing this. Realistically, RAG is going to be an important part of many applications, and if there is an application that needs to access lots of data, but it not used that often, this would be a great opportunity for something like the Kioxia AiSAQ.”

Must Read

Leave a Comment Cancel Reply