AI · Official
NVIDIA Guide Shows How to Design Sparse Attention for Faster Long-Context AI
NVIDIA Technical blog ·

At a glance
NVIDIA published a practical guide for model developers on designing sparse attention, a technique that lets AI models focus on only a small part of a long context. The post explains how the parts of the design affect speed on NVIDIA GPUs. It uses a production FP8 kernel to test design choices.
Key points
- Long inputs of hundreds of thousands of tokens make standard attention the most expensive part of a model layer.
- Sparse attention lets each query look at only a few thousand keys, but an indexer that picks those keys still scores every key.
- In the DeepSeek V3.2 example, the indexer and Top-k selection take 47% of the prefill layer time.
- In decode, more query heads add computation but not memory reads, so runtime barely changes.
Why it matters
Long documents and large codebases make AI slow and costly, so these design choices decide how fast models can run on real hardware.
Original source
Co-Designing Sparse Attention for Efficient Long-Context Inference
NVIDIA Technical blog · Nov 18, 02:30 KST
Read on NVIDIA Technical blog ↗AI summarized the original article. Check the original for accuracy. Copyright belongs to NVIDIA Technical blog.
More from this field
- Amazon Releases SMART Toolkit to Simplify WhatsApp AI Agents on AgentCore
Amazon has released SMART, an open source Python toolkit and Terraform setup that handles the WhatsApp connection for AI agents built on Amazon Bedrock AgentCore. Developers keep control of their agent's logic while SMART takes care of messages, media, memory, and scheduling. The article says one command deploys the whole infrastructure to an AWS account.
Amazon AWS Machine Learning blog · Nov 18, 03:48 KST - Google Extends Gemini Chat Retention to Two Months, Even With History Off
Google is extending how long it keeps temporary Gemini chats, including those made with chat history turned off, to about two months. Before, it kept them for 72 hours. Google says the change is meant to improve safety and security as AI systems grow more capable.
TechCrunch · Nov 18, 03:48 KST - OpenAI Demos GPT-6.2 Astra Rebuilding a 3D Room From Two Video Frames
OpenAI shows how its GPT-6.2 Astra model can recreate a room from just two video frames. The model builds the set in Blender, changes the decor and lighting, and turns it into an interactive space and a virtual webcam background. The video is a demo of the model's coding and 3D skills.
OpenAI 공식 YouTube · Nov 18, 03:41 KST - Edge for Business Adds Agentic Browsing and Skills Recorder at Ignite 2026
Microsoft shared new features for its Edge for Business browser at its Ignite 2026 conference. Agentic browsing, which lets AI agents finish multi-step tasks across websites, is now part of Cowork and is generally available. A skills recorder for turning workflows into reusable steps is coming in preview, and faster agent performance is being tested.
Microsoft Edge blog · Nov 18, 03:40 KST