NVIDIA Adds Automatic AI Workload Inventory to Kubernetes GPU Tooling

At a glance

NVIDIA added an AI workload inventory feature to its GPU platform tool AICR. It records which AI workloads actually run on Kubernetes (a tool that manages containers across many servers). The feature is built on the open-source k8s-aibom project from Google.

Key points

1. 1

   It automatically creates an AI bill of materials (AIBOM), a list of the components AI workloads use.

2. 2

   It is turned on by default in two default configurations for H100 GPUs on Google Kubernetes Engine (GKE).

3. 3

   Other GKE configurations can turn it on or off with a command option.

4. 4

   In testing, it sent about four requests per minute to the server, and CPU and memory use stayed low.

Why it matters

Containers with the same image tag can run different code on different nodes, so checking which AI workloads are actually running matters for security and management.