AI · Article
SlipStream Speeds Up Flash Memory for GPU AI Models Using Smart Prefetching
Semiconductor Engineering ·

At a glance
Researchers from Georgia Tech, NVIDIA Research, and Stanford University published a paper called SlipStream. It is a way to make High-Bandwidth Flash (HBF) work well as memory for GPU systems running large language models. Flash memory can hold much more data than DRAM-based memory, but it responds much more slowly.
Key points
- HBF uses NAND flash in place of the DRAM chips in HBM, giving up to 16 times more capacity at similar read speed.
- HBF can take nearly 20 microseconds to read, about 20 times slower than HBM, which can slow AI inference by up to 12.8 times.
- SlipStream loads data in the order the model needs it, using a staging process that starts fetching data before it is requested.
- In tests across seven dense models, SlipStream with a 40 MB buffer reached 92.4% of the performance of an ideal HBF with HBM-like latency.
Why it matters
Large AI models need more memory than GPUs can hold, so making cheaper, larger flash memory fast enough could lower the cost of running them.
Original source
Reducing High-Bandwidth Flash Latency In GPU Systems (Georgia Tech, Nvidia, Stanford)
Semiconductor Engineering · Nov 17, 08:52 KST
Read on Semiconductor Engineering ↗AI summarized the original article. Check the original for accuracy. Copyright belongs to Semiconductor Engineering.
More from this field
- PostNord Swaps Its Phone Support AI for ElevenAgents, Lifting Call Resolution to 60%
PostNord, the leading postal and logistics company in the Nordics, replaced its AI phone system in Sweden with a new one built on ElevenLabs' ElevenAgents. The old system was fully shut down in about 90 days, and the share of calls resolved without escalation rose from 45% to 60%.
ElevenLabs blog · Nov 17, 21:00 KST - Nvidia Scandal Suggests Trump Administration Put China Race Over US Security
A New York Times opinion piece argues that the Nvidia scandal shows the Trump administration ignored national security while claiming to win the AI race with China. The writer says the administration loosened AI rules, often citing that race. The article text was partly cut off, so the summary covers only the visible part.
The New York Times · Nov 17, 10:18 KST - Laon Secure Links Google Device Trust to Its Zero-Trust Security Products
Laon Secure, a Korean AI security and authentication company, announced that it has linked Google's Device Trust to its core authentication and device management products. It also joined Google's Android Enterprise partner program.
머니투데이 · Nov 17, 09:02 KST - Five Types of AI Software Factories, Explained with Pros and Cons
This video explains five common types of AI software factories, which are systems that use AI to help build software. It covers what each type is and the pros and cons of each, so viewers can choose the right balance between human control and AI autonomy.
Cole Medin · Nov 17, 08:51 KST