Back to news

AI · Official

AWS Shows How NVIDIA Run:ai Lets Pods Share One GPU's Memory on Amazon EKS

Amazon AWS Machine Learning blog ·

At a glance

AWS published a guide on running NVIDIA Run:ai GPU Fractioning on Amazon EKS so several workloads can share one GPU. In a test on one A10G GPU, two Pods each got 6 GiB within an 8 GiB limit, and an allocation above that limit was rejected. The method adds enforced GPU-memory limits and per-Pod metrics to software-based GPU sharing.

Key points

Why it matters

Teams can run notebooks and small services on one GPU without waiting for a whole GPU, but the results do not prove faster throughput or lower cost.

Original source

Using NVIDIA Run:ai GPU Fractioning on Amazon EKS

Amazon AWS Machine Learning blog · Nov 20, 01:13 KST

Read on Amazon AWS Machine Learning blog ↗

AI summarized the original article. Check the original for accuracy. Copyright belongs to Amazon AWS Machine Learning blog.

AI