AI · អត្ថបទ
AWS Adds NVIDIA GPU Partitioning to SageMaker Inference Endpoints to Cut Costs
Unite.AI ·

មើលត្រួសៗ
AWS now lets a single GPU be split into several pieces on SageMaker real-time endpoints, so multiple models can share it. This uses resources more efficiently than giving an entire GPU to one model. However, it does not benefit every workload.
ចំណុចសំខាន់ៗ
- A GPU is divided into several hardware-isolated partitions, and each model gets dedicated compute and memory.
- This works only on real-time endpoints that use inference components, and does not apply to serverless or asynchronous modes.
- In AWS tests, H100 partitions performed better in output speed and response latency when inputs were long.
- For short inputs or low numbers of concurrent requests, using a whole A100 GPU was still cheaper.
ហេតុអ្វីវាសំខាន់
When serving AI models, GPU costs could drop significantly, which may help companies run AI more cheaply.
ប្រភពដើម
AWS, SageMaker 추론 엔드포인트에 NVIDIA GPU 분할 기능 추가
Unite.AI · Nov 7, 03:23 KST
Read on Unite.AI ↗AI summarized the original article. Check the original for accuracy. Copyright belongs to Unite.AI.
Amazon Web Services · ចំណាត់ថ្នាក់ និងព័ត៌មានបន្ថែម
បន្ថែមពីវិស័យនេះ
- AWS Marketplace Lets AI Agents Buy Software on Their Own
AWS has updated its AWS Marketplace so AI agents can buy software, APIs, data, and content on their own, not just find them. Buyers pay in dollars, and charges are added to their AWS bill.
디지털투데이 · Nov 21, 07:37 KST - Amazon SageMaker HyperPod Global Inference Gateway Routes AI Requests Across Regions
Amazon SageMaker HyperPod now has a Global Inference Gateway that sends each request to the best cluster across regions through one endpoint. Before this, clients had to choose a cluster themselves, and a busy cluster could leave requests waiting while idle GPUs sat elsewhere.
Amazon AWS Machine Learning blog · Nov 21, 07:35 KST - Google Drops Search Button and Adds AI Sidebar to Its Search Homepage
Google has officially rolled out a redesigned desktop search homepage after a successful test. The Google Search button next to the search bar is gone, and a row of AI buttons now sits under the bar. Logged-in users also get a sidebar showing their recent AI Mode conversations.
Ars Technica · Nov 21, 06:58 KST - Meta Rethinks 2030 Net-Zero Goal as AI Data Center Emissions Hit Record High
Meta's 2026 sustainability report said the company is re-evaluating its 2030 net-zero goal. Its emissions rose about 40 percent from 2024 to 2025, the largest jump among the big US tech companies.
Ars Technica · Nov 21, 06:47 KST