KAI Scheduler

KAI Scheduler is an open source Kubernetes scheduler for AI and machine learning workloads, developed under the kai-scheduler GitHub organization and written in Go. It lets cluster administrators dynamically allocate GPU resources across teams and workload types, from small interactive jobs to large training and inference. The README lists v0.18.5 as the latest release and states the project is a Cloud Native Computing Foundation sandbox project.

What It Is

KAI Scheduler is a Kubernetes scheduler built on kube-batch. It is designed for large GPU clusters with thousands of nodes and high workload throughput. It can run alongside other schedulers installed on the same cluster, and it works with cloud autoscalers such as Karpenter as well as static on-premise deployments.

Scheduling and Fairness Capabilities

The scheduler provides batch (gang) scheduling, bin-packing and spread strategies, workload priority, and hierarchical queues with quotas, limits and over-quota weights. Fairness is handled through Dominant Resource Fairness, reclaim strategies, and time-based fairshare that accounts for historical usage. Further documented features include a minimum guaranteed runtime, workload consolidation, elastic workloads, background pods, and GPU sharing so multiple workloads can use the same GPUs.

Topology and Hardware Support

Topology-Aware Scheduling and Hierarchical PodGroups handle placement for distributed and disaggregated workloads, and are integrated with Grove and Dynamo. Dynamic Resource Allocation through Kubernetes ResourceClaims supports vendor-specific hardware, including NVIDIA GB200/GB300. The project also lists native KubeRay integration for Ray workloads, scheduler explainability through Kubernetes Events, and workload signatures for large multi-pod submissions.

Setup Path

Installation requires a running Kubernetes cluster and the Helm CLI. The NVIDIA GPU Operator is needed to schedule workloads that request GPUs. KAI Scheduler installs into the kai-scheduler namespace via an OCI Helm chart, from source, or with ArgoCD for GitOps. OpenShift has specific flag instructions, and the repository includes a quick-start example. The project also publishes scale-test and benchmark dashboards.