Skip to the content
Know your tech before you buy it. Makers: get listed from $49/mo

KAI Scheduler

kai-scheduler.dev

  • Free plan
  • Self-hosted
6.9 Club score Visit kai-scheduler.dev

What KAI Scheduler is

KAI Scheduler is a free, Apache-2.0 Kubernetes scheduler that allocates GPUs for AI data processing, training, and inference. It targets large clusters and supports workloads from interactive notebooks to multi-node distributed jobs. Hierarchical queue trees provide quotas, limits, priorities, and borrowing between teams. Gang scheduling places distributed pods together, while elastic workloads adjust replica counts within configured minimum and maximum values as capacity changes. GPU sharing supports time slicing, MPS, and MIG. Additional scheduling includes topology-aware placement, time-based fairshare, and priority-based preemption. Native integrations are documented for Ray on Kubernetes and Grove for complex workloads. KAI can run in cloud or on-premises environments, including Karpenter autoscaling. Installation uses a production Helm chart, source builds, or ArgoCD, and requires a running Kubernetes cluster, Helm, and NVIDIA GPU Operator for GPU requests. The project is a CNCF sandbox project. LTS releases receive one year of public support. In the default NonMemoryEnforced sharing mode, KAI does not enforce runtime GPU memory isolation.

Who it suits

It suits Kubernetes teams scheduling shared GPUs across cloud or on-premises clusters. It is aimed at organizations running mixed interactive, batch, and distributed AI workloads.

Good to know, and what to watch

  • Hierarchical queues provide quotas and borrowing
  • Supports gang and elastic scheduling
  • Offers time slicing, MPS, and MIG sharing
  • Includes topology-aware placement and fairshare
  • Runs in cloud or on-premises environments
  • Requires Kubernetes and Helm
  • GPU workloads require NVIDIA GPU Operator
  • Default sharing mode lacks runtime memory isolation
  • LTS public support lasts one year

The verdict

KAI Scheduler provides detailed queue, fairness, placement, and GPU-sharing controls for Kubernetes AI clusters. Plan for its infrastructure prerequisites and the memory-isolation limitation in shared GPU deployments.

What the maker publishes

Cloud and on-premises
The project describes compatibility with dynamic cloud infrastructure, including Karpenter autoscalers, and static on-premises deployments. source
Deployment
KAI Scheduler can be installed from a production Helm chart, built from source, or installed with ArgoCD; it requires a running Kubernetes cluster and Helm. source
Deployment
The project provides production Helm installation, source build, and ArgoCD GitOps installation options for Kubernetes. source
Elastic workloads
Elastic workloads can grow or shrink within configured minimum and maximum replica counts as capacity changes. source
Fairness
Time-based fairshare tracks historical GPU usage over a configurable window to distribute over-quota resources fairly over time. source
Gang and elastic scheduling
Gang scheduling places distributed workloads all at once, while elastic workloads can grow or shrink within minimum and maximum replica counts as capacity allows. source
Gang scheduling
Gang scheduling places all pods in a distributed workload together or waits until resources are available. source
GPU sharing
GPU sharing supports time slicing, MPS, and MIG so multiple workloads can share GPU devices. source
GPU sharing
GPU sharing supports time slicing, MPS, and MIG so workloads can share GPU devices. source
GPU sharing limitation
The GPU-sharing documentation says the default NonMemoryEnforced mode schedules fractional GPU workloads without KAI enforcing runtime GPU memory isolation. source
Hierarchical queues
Multi-level queue trees support quotas, limits, and over-quota borrowing across teams. source
Integrations
The project documents native integration for Ray workloads on Kubernetes and integration with Grove for complex workloads. source
Integrations
The project states it is natively integrated with KubeRay for Ray workloads on Kubernetes and describes integrations with Grove and Dynamo for complex multi-component workloads. source
Intended workloads
It supports workloads ranging from interactive notebooks to multi-node distributed training in the same cluster. source
License
The public GitHub repository identifies the project as open source and lists an Apache-2.0 license. source
License and project
KAI Scheduler is an open-source Apache-2.0 project and a CNCF sandbox project. source
Prerequisite
The installation guide says NVIDIA GPU-Operator is required to schedule workloads requesting GPU resources. source
Prerequisites
The installation guide lists a running Kubernetes cluster and Helm CLI, and requires NVIDIA GPU Operator for workloads requesting GPU resources. source
Priority and preemption
Per-queue and per-workload priorities let critical jobs preempt while non-critical jobs back off. source
Purpose
KAI Scheduler is a Kubernetes-native scheduler that optimizes GPU allocation for AI workloads across data processing, training, and inference. source
Purpose
KAI Scheduler is a Kubernetes-native scheduler for AI workloads that optimizes GPU allocation across data processing, training, and inference while keeping resource shares fair across teams. source
Queues
Hierarchical queues support quotas, limits, priorities, and borrowing resources across teams. source
Scale
The project says it is designed and continuously tested for large GPU clusters with thousands of nodes and high-throughput workloads. source
Security
Security vulnerabilities should be reported through GitHub Security Advisories rather than public GitHub issues. source

Checked against the maker's pages on 7 Oct 2026.