AI · ផ្លូវការ
NVIDIA Guide Shows How to Design Sparse Attention for Faster Long-Context AI
NVIDIA Technical blog ·

មើលត្រួសៗ
NVIDIA published a practical guide for model developers on designing sparse attention, a technique that lets AI models focus on only a small part of a long context. The post explains how the parts of the design affect speed on NVIDIA GPUs. It uses a production FP8 kernel to test design choices.
ចំណុចសំខាន់ៗ
- Long inputs of hundreds of thousands of tokens make standard attention the most expensive part of a model layer.
- Sparse attention lets each query look at only a few thousand keys, but an indexer that picks those keys still scores every key.
- In the DeepSeek V3.2 example, the indexer and Top-k selection take 47% of the prefill layer time.
- In decode, more query heads add computation but not memory reads, so runtime barely changes.
ហេតុអ្វីវាសំខាន់
Long documents and large codebases make AI slow and costly, so these design choices decide how fast models can run on real hardware.
ប្រភពដើម
Co-Designing Sparse Attention for Efficient Long-Context Inference
NVIDIA Technical blog · Nov 18, 02:30 KST
Read on NVIDIA Technical blog ↗AI summarized the original article. Check the original for accuracy. Copyright belongs to NVIDIA Technical blog.
បន្ថែមពីវិស័យនេះ
- Designers Say Ssense's Website Shows Their Clothes Inaccurately
Designer Reese Cooper noticed his RCOS puffer coat looked distorted on the luxury retailer Ssense's website, with a chest hook drawn far too large. Ssense said some site images came from a "virtual try-on" program, though it would not say whether that program uses AI.
The New York Times · Nov 21, 06:16 KST - Walter Writes AI Tool Rewrites AI Text to Sound More Human, Ad Test Shows
This short video is a sponsored ad for Walter Writes AI, a tool that rewrites AI-generated text so it reads more like a person wrote it. The creator says they ran a paragraph through its detector, rewrote the flagged parts, and compared the scores before and after.
TheAIGRID · Nov 21, 06:15 KST - Korean Firms Build Three-Way AI Push: Anthropic Stake, HBM Chips, Claude Rollout
Korean companies are expanding their roles in the global AI market around Anthropic's IPO. SK Telecom earned large paper gains from an early investment, while Samsung Electronics and SK hynix are strengthening chip supply ties. Samsung SDS and LG CNS are bringing Anthropic's Claude model into the business AI market.
뉴스웨이 · Nov 21, 06:10 KST - MIT and IBM Team Probes Gaps in Multimodal AI Understanding
MIT and IBM researchers are studying where today's multimodal AI models fall short, even when they handle text, images, and sound fluently. Their team, co-led by MIT's Jim Glass, builds new benchmarks and training methods that need less labeled data. This Q&A walks through how the project started and how it has evolved.
MIT News · Nov 21, 06:00 KST