diff --git a/ai-tooling/user/skills/nvcf-self-managed-prerequisite/SKILL.md b/ai-tooling/user/skills/nvcf-self-managed-prerequisite/SKILL.md index f44514fb3..52e787ac9 100644 --- a/ai-tooling/user/skills/nvcf-self-managed-prerequisite/SKILL.md +++ b/ai-tooling/user/skills/nvcf-self-managed-prerequisite/SKILL.md @@ -35,7 +35,7 @@ One operator tool plus two cluster-level components the NVCA operator / compute | Prereq | Why it is needed | Detail | | ------ | ------------------ | ------ | | `nvcf-cli` | The compute-plane stack's `make register-cluster` (and `install`/`apply`/`sync`, which abort without the registration values it writes) shells out to `nvcf-cli`. The shipped stack defaults to building it from a sibling `../cli` checkout that the release does not include, so a green-field repo fails with `ensure-nvcf-cli` / "Registration values not found". | See Step 0b below | -| KAI Scheduler | `selfManaged.featureGateValues` includes `KAIScheduler`; NVCA polls `Queue` CRs and refuses to become healthy until their quotas are `-1` | [references/kai-scheduler.md](references/kai-scheduler.md) | +| KAI Scheduler | `selfManaged.featureGateValues` includes `KAIScheduler`; NVCA polls `Queue` CRs and refuses to become healthy until their quotas are `-1`. Skip the standalone helm step below when `addons.kaiScheduler.enabled` is set in `nvcf-compute-plane` (required when enabling Grove or Dynamo). | [references/kai-scheduler.md](references/kai-scheduler.md) | | SMB CSI driver (`smb.csi.k8s.io`) | NVCA's `selfManaged.sharedStorage` runs Samba sidecar pods that export file shares; the resulting PVCs need this CSI driver to bind | [references/smb-csi.md](references/smb-csi.md) | The KAI Scheduler and SMB CSI installs are cloud-neutral helm commands pinned to NVCF-validated versions. These are upstream third-party charts (not NVCF images), so they are not in `manifest.yaml`; the per-component reference docs carry the current pin and link the NVCF docs version table. `nvcf-cli` is an operator workstation tool, not an in-cluster install. @@ -82,6 +82,8 @@ make install CLUSTER_NAME= HELMFILE_ENV= ### 1 — KAI Scheduler +Skip this step when `addons.kaiScheduler.enabled` is true in `nvcf-compute-plane`. That stack installs KAI Scheduler (release and namespace `kai-scheduler`). Enable that flag whenever Grove or Dynamo is enabled. Use the standalone install below when you need KAI without the compute-plane add-on (for example the NVCA `KAIScheduler` feature gate alone). + ```bash cat > nvca-values.yaml << 'EOF' scheduler: diff --git a/ai-tooling/user/skills/nvcf-self-managed-prerequisite/references/kai-scheduler.md b/ai-tooling/user/skills/nvcf-self-managed-prerequisite/references/kai-scheduler.md index 3710c9a15..9c24d82a4 100644 --- a/ai-tooling/user/skills/nvcf-self-managed-prerequisite/references/kai-scheduler.md +++ b/ai-tooling/user/skills/nvcf-self-managed-prerequisite/references/kai-scheduler.md @@ -14,6 +14,8 @@ At the time of writing, the pinned chart version is `v0.14.0`. See [NVCF KAI Sch ## Install +When `addons.kaiScheduler.enabled` is true in `nvcf-compute-plane`, that stack installs KAI Scheduler for you. Skip this section in that case. + ```bash cat > nvca-values.yaml << 'EOF' scheduler: diff --git a/deploy/stacks/nvcf-compute-plane/Makefile.dist b/deploy/stacks/nvcf-compute-plane/Makefile.dist index c57fac7c3..4619ffb2e 100644 --- a/deploy/stacks/nvcf-compute-plane/Makefile.dist +++ b/deploy/stacks/nvcf-compute-plane/Makefile.dist @@ -35,7 +35,7 @@ REQUIRED_HELMFILE_MINOR := 1 REQUIRED_HELM_MAJOR := 3 # Namespaces owned by the compute-plane stack (used by destroy target) -NAMESPACES := nvca-operator grove-system dynamo-system +NAMESPACES := nvca-operator grove-system dynamo-system kai-scheduler # Kubeconfig file targeting a specific cluster. Used by helmfile, kubectl, # and nvcf-cli. Required in multi-cluster workflows to avoid registering diff --git a/deploy/stacks/nvcf-compute-plane/README.md b/deploy/stacks/nvcf-compute-plane/README.md index d0ec1639b..2ac979a0d 100644 --- a/deploy/stacks/nvcf-compute-plane/README.md +++ b/deploy/stacks/nvcf-compute-plane/README.md @@ -89,17 +89,28 @@ make install CLUSTER_NAME=... # downloads helmfile v1.1.9 + helm v3.15.4 on fi ## Optional Add-ons -Grove (topology-aware scheduling) and Dynamo (inference framework scheduling) are disabled -by default. Enable per-environment in `environments/.yaml`: +KAI Scheduler, Grove (topology-aware scheduling), and Dynamo (inference framework +scheduling) are disabled by default. Grove and Dynamo require KAI Scheduler +(release and namespace `kai-scheduler`). Enable per-environment in +`environments/.yaml`: ```yaml addons: + kaiScheduler: + enabled: true groveOperator: enabled: true dynamoOperator: enabled: true ``` +Override KAI component resources under `addons.kaiScheduler..resources` +(for example `addons.kaiScheduler.scheduler.resources.requests.memory`). Defaults +are set in `helmfile.d/01-dependencies.yaml.gotmpl`. + +For a standalone KAI install outside this stack, follow the +[KAI Scheduler guide](https://docs.nvidia.com/cloud-functions/current/latest/cluster-management/kai-scheduler.html). + ## Multi-Cluster Example Each cluster is registered and installed independently. Pass `KUBECONFIG_FILE` diff --git a/deploy/stacks/nvcf-compute-plane/environments/base.yaml b/deploy/stacks/nvcf-compute-plane/environments/base.yaml index 1062383ee..b987ee657 100644 --- a/deploy/stacks/nvcf-compute-plane/environments/base.yaml +++ b/deploy/stacks/nvcf-compute-plane/environments/base.yaml @@ -72,6 +72,12 @@ observability: # ============================================================================= addons: + # KAI Scheduler (disabled by default). Required when enabling Grove or Dynamo. + # Per-component resource overrides: addons.kaiScheduler..resources + # (requests/limits). Defaults live in helmfile.d/01-dependencies.yaml.gotmpl. + kaiScheduler: + enabled: false + # Grove topology-aware scheduling operator (disabled by default) groveOperator: enabled: false diff --git a/deploy/stacks/nvcf-compute-plane/helmfile.d/01-dependencies.yaml.gotmpl b/deploy/stacks/nvcf-compute-plane/helmfile.d/01-dependencies.yaml.gotmpl index 4379f451c..37df947a8 100644 --- a/deploy/stacks/nvcf-compute-plane/helmfile.d/01-dependencies.yaml.gotmpl +++ b/deploy/stacks/nvcf-compute-plane/helmfile.d/01-dependencies.yaml.gotmpl @@ -1,6 +1,7 @@ # NVCF Compute Plane Releases # -# Two optional releases — installed once per GPU cluster: +# Optional releases — installed once per GPU cluster: +# kai-scheduler — optional; addons.kaiScheduler.enabled (required by grove/dynamo) # grove-operator — optional; topology-aware scheduling (addons.groveOperator.enabled) # dynamo-operator — optional; Dynamo inference framework scheduling (addons.dynamoOperator.enabled) # @@ -16,6 +17,9 @@ environments: --- repositories: + - name: kai-scheduler + url: ghcr.io/kai-scheduler/kai-scheduler + oci: true - name: nvidia-ai-dynamo-grove url: ghcr.io/ai-dynamo/grove oci: true @@ -33,6 +37,197 @@ helmDefaults: releases: + - name: kai-scheduler + # Required by grove-operator and dynamo-operator. Release name and namespace + # must be "kai-scheduler" per KAI installation requirements. + # Enable per-environment: addons.kaiScheduler.enabled: true + # Per-component resources can be overridden under addons.kaiScheduler..resources; + # defaults below match the managed SBOM template. + condition: addons.kaiScheduler.enabled + chart: kai-scheduler/kai-scheduler + version: v0.14.0 + namespace: kai-scheduler + values: + - ../global.yaml.gotmpl + - {{- $kai := dig "kaiScheduler" dict .Values.addons }} + {{- $opRes := dig "operator" "resources" dict $kai }} + {{- $opReq := dig "requests" dict $opRes }} + {{- $opLim := dig "limits" dict $opRes }} + {{- $pgRes := dig "podgrouper" "resources" dict $kai }} + {{- $pgReq := dig "requests" dict $pgRes }} + {{- $pgLim := dig "limits" dict $pgRes }} + {{- $pgcRes := dig "podgroupcontroller" "resources" dict $kai }} + {{- $pgcReq := dig "requests" dict $pgcRes }} + {{- $pgcLim := dig "limits" dict $pgcRes }} + {{- $bdRes := dig "binder" "resources" dict $kai }} + {{- $bdReq := dig "requests" dict $bdRes }} + {{- $bdLim := dig "limits" dict $bdRes }} + {{- $schRes := dig "scheduler" "resources" dict $kai }} + {{- $schReq := dig "requests" dict $schRes }} + {{- $schLim := dig "limits" dict $schRes }} + {{- $qcRes := dig "queuecontroller" "resources" dict $kai }} + {{- $qcReq := dig "requests" dict $qcRes }} + {{- $qcLim := dig "limits" dict $qcRes }} + {{- $adRes := dig "admission" "resources" dict $kai }} + {{- $adReq := dig "requests" dict $adRes }} + {{- $adLim := dig "limits" dict $adRes }} + {{- $nsaRes := dig "nodescaleadjuster" "resources" dict $kai }} + {{- $nsaReq := dig "requests" dict $nsaRes }} + {{- $nsaLim := dig "limits" dict $nsaRes }} + {{- $npeRes := dig "numaPlacementExporter" "resources" dict $kai }} + {{- $npeReq := dig "requests" dict $npeRes }} + {{- $npeLim := dig "limits" dict $npeRes }} + global: + securityContext: + runAsUser: 10000 + runAsNonRoot: true + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + {{- if .Values.global.nodeSelector.enabled }} + nodeSelector: + {{ .Values.global.nodeSelector.key }}: {{ .Values.global.nodeSelector.value }} + {{- end }} + {{- if .Values.global.imagePullSecrets }} + imagePullSecrets: + {{- toYaml .Values.global.imagePullSecrets | nindent 12 }} + {{- end }} + {{- if not (empty .Values.global.tolerations) }} + tolerations: + {{- toYaml .Values.global.tolerations | nindent 12 }} + {{- end }} + + # Image names stay at chart defaults (ghcr.io/kai-scheduler/kai-scheduler/*). + # Managed SBOM uses kai-scheduler-* names with nv-ngc-devops; those do not + # resolve against the public chart registry. + operator: + resources: + requests: + cpu: {{ dig "cpu" "50m" $opReq }} + memory: {{ dig "memory" "256Mi" $opReq }} + limits: + cpu: {{ dig "cpu" "1000m" $opLim }} + memory: {{ dig "memory" "1Gi" $opLim }} + + podgrouper: + resources: + requests: + cpu: {{ dig "cpu" "50m" $pgReq }} + memory: {{ dig "memory" "512Mi" $pgReq }} + limits: + cpu: {{ dig "cpu" "1000m" $pgLim }} + memory: {{ dig "memory" "4Gi" $pgLim }} + + podgroupcontroller: + resources: + requests: + cpu: {{ dig "cpu" "50m" $pgcReq }} + memory: {{ dig "memory" "256Mi" $pgcReq }} + limits: + cpu: {{ dig "cpu" "1000m" $pgcLim }} + memory: {{ dig "memory" "1Gi" $pgcLim }} + + binder: + resources: + requests: + cpu: {{ dig "cpu" "50m" $bdReq }} + memory: {{ dig "memory" "2Gi" $bdReq }} + limits: + cpu: {{ dig "cpu" "4000m" $bdLim }} + memory: {{ dig "memory" "4Gi" $bdLim }} + + scheduler: + placementStrategy: binpack + plugins: + nodeplacement: + arguments: + gpu: binpack + cpu: spread + actions: + preempt: + enabled: false + consolidation: + enabled: false + resources: + requests: + cpu: {{ dig "cpu" "500m" $schReq }} + memory: {{ dig "memory" "3Gi" $schReq }} + limits: + cpu: {{ dig "cpu" "2000m" $schLim }} + memory: {{ dig "memory" "6Gi" $schLim }} + + queuecontroller: + resources: + requests: + cpu: {{ dig "cpu" "50m" $qcReq }} + memory: {{ dig "memory" "256Mi" $qcReq }} + limits: + cpu: {{ dig "cpu" "1000m" $qcLim }} + memory: {{ dig "memory" "1Gi" $qcLim }} + + admission: + resources: + requests: + cpu: {{ dig "cpu" "50m" $adReq }} + memory: {{ dig "memory" "256Mi" $adReq }} + limits: + cpu: {{ dig "cpu" "1000m" $adLim }} + memory: {{ dig "memory" "1Gi" $adLim }} + + nodescaleadjuster: + resources: + requests: + cpu: {{ dig "cpu" "50m" $nsaReq }} + memory: {{ dig "memory" "256Mi" $nsaReq }} + limits: + cpu: {{ dig "cpu" "1000m" $nsaLim }} + memory: {{ dig "memory" "1Gi" $nsaLim }} + + numaPlacementExporter: + resources: + requests: + cpu: {{ dig "cpu" "50m" $npeReq }} + memory: {{ dig "memory" "64Mi" $npeReq }} + limits: + cpu: {{ dig "cpu" "1000m" $npeLim }} + memory: {{ dig "memory" "1Gi" $npeLim }} + + defaultQueue: + createDefaultQueue: true + parentName: default-parent-queue + childName: default-queue + parentResources: + cpu: + quota: -1 + limit: -1 + overQuotaWeight: 1 + gpu: + quota: -1 + limit: -1 + overQuotaWeight: 1 + memory: + quota: -1 + limit: -1 + overQuotaWeight: 1 + childResources: + cpu: + quota: -1 + limit: -1 + overQuotaWeight: 1 + gpu: + quota: -1 + limit: -1 + overQuotaWeight: 1 + memory: + quota: -1 + limit: -1 + overQuotaWeight: 1 + labels: + release-group: workers + wait: true + waitForJobs: true + - name: grove-operator # Topology-aware scheduling operator (optional; disabled by default). # Enable per-environment: addons.groveOperator.enabled: true @@ -40,6 +235,8 @@ releases: chart: nvidia-ai-dynamo-grove/grove-charts version: v0.1.0-alpha.9 namespace: grove-system + needs: + - kai-scheduler/kai-scheduler values: - ../global.yaml.gotmpl - replicaCount: 1 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/environments/local.yaml b/deploy/stacks/nvcf-compute-plane/testdata/environments/local.yaml index e3070879b..f5ecdef06 100644 --- a/deploy/stacks/nvcf-compute-plane/testdata/environments/local.yaml +++ b/deploy/stacks/nvcf-compute-plane/testdata/environments/local.yaml @@ -18,6 +18,8 @@ global: natsURL: nats://nats.nats-system.svc.cluster.local:4222 addons: + kaiScheduler: + enabled: true groveOperator: enabled: true dynamoOperator: diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/component-serviceaccount.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/component-serviceaccount.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/component-serviceaccount.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/component-serviceaccount.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/deployment.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/deployment.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/deployment.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/deployment.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/epp.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/epp.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/epp.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/epp.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/gpu-discovery-preflight.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/gpu-discovery-preflight.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/gpu-discovery-preflight.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/gpu-discovery-preflight.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/gpu-discovery-rbac.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/gpu-discovery-rbac.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/gpu-discovery-rbac.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/gpu-discovery-rbac.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/leader-election-rbac.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/leader-election-rbac.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/leader-election-rbac.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/leader-election-rbac.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/manager-rbac.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/manager-rbac.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/manager-rbac.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/manager-rbac.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/metrics-auth-rbac.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/metrics-auth-rbac.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/metrics-auth-rbac.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/metrics-auth-rbac.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/metrics-service.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/metrics-service.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/metrics-service.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/metrics-service.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/operator-config.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/operator-config.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/operator-config.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/operator-config.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/operator-servicemonitor.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/operator-servicemonitor.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/operator-servicemonitor.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/operator-servicemonitor.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/planner.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/planner.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/planner.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/planner.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/profiling-job-rbac.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/profiling-job-rbac.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/profiling-job-rbac.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/profiling-job-rbac.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/prometheus.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/prometheus.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/prometheus.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/prometheus.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/regcred-secret.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/regcred-secret.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/regcred-secret.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/regcred-secret.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/serviceaccount.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/serviceaccount.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/serviceaccount.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/serviceaccount.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/validate-values.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/validate-values.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/validate-values.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/validate-values.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/webhook-certificates.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/webhook-certificates.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/webhook-certificates.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/webhook-certificates.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/webhook-configuration.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/webhook-configuration.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/webhook-configuration.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/webhook-configuration.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/webhook-rbac.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/webhook-rbac.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/webhook-rbac.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/webhook-rbac.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/webhook-service.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/webhook-service.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/webhook-service.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/dynamo-operator/templates/webhook-service.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/nats/templates/config-map.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/nats/templates/config-map.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/nats/templates/config-map.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/nats/templates/config-map.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/nats/templates/headless-service.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/nats/templates/headless-service.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/nats/templates/headless-service.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/nats/templates/headless-service.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/nats/templates/pod-disruption-budget.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/nats/templates/pod-disruption-budget.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/nats/templates/pod-disruption-budget.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/nats/templates/pod-disruption-budget.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/nats/templates/service.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/nats/templates/service.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/nats/templates/service.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/nats/templates/service.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/nats/templates/stateful-set.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/nats/templates/stateful-set.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/charts/nats/templates/stateful-set.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/charts/nats/templates/stateful-set.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/templates/kai.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/templates/kai.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/templates/kai.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/templates/kai.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/templates/validate-values.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/templates/validate-values.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-dynamo-operator/dynamo-platform/templates/validate-values.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-dynamo-operator/dynamo-platform/templates/validate-values.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/clusterrole.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/clusterrole.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/clusterrole.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/clusterrole.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/clusterrolebinding.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/clusterrolebinding.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/clusterrolebinding.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/clusterrolebinding.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/clustertopology-validating-webhook-config.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/clustertopology-validating-webhook-config.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/clustertopology-validating-webhook-config.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/clustertopology-validating-webhook-config.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/configmap-operator.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/configmap-operator.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/configmap-operator.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/configmap-operator.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/deployment.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/deployment.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/deployment.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/deployment.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/leaderelection-role.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/leaderelection-role.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/leaderelection-role.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/leaderelection-role.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/leaderelection-rolebinding.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/leaderelection-rolebinding.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/leaderelection-rolebinding.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/leaderelection-rolebinding.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/pcs-defaulting-webhook-config.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/pcs-defaulting-webhook-config.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/pcs-defaulting-webhook-config.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/pcs-defaulting-webhook-config.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/pcs-validating-webhook-config.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/pcs-validating-webhook-config.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/pcs-validating-webhook-config.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/pcs-validating-webhook-config.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/service.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/service.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/service.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/service.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/serviceaccount.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/serviceaccount.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-83748d4e-grove-operator/grove-charts/templates/serviceaccount.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-grove-operator/grove-charts/templates/serviceaccount.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/default-queue.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/default-queue.yaml new file mode 100644 index 000000000..89102db45 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/default-queue.yaml @@ -0,0 +1,47 @@ +--- +# Source: kai-scheduler/templates/default-queue.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 +apiVersion: scheduling.run.ai/v2 +kind: Queue +metadata: + name: default-parent-queue + annotations: + helm.sh/resource-policy: keep +spec: + resources: + cpu: + quota: -1 + limit: -1 + overQuotaWeight: 1 + gpu: + quota: -1 + limit: -1 + overQuotaWeight: 1 + memory: + quota: -1 + limit: -1 + overQuotaWeight: 1 +--- +# Source: kai-scheduler/templates/default-queue.yaml +apiVersion: scheduling.run.ai/v2 +kind: Queue +metadata: + name: default-queue + annotations: + helm.sh/resource-policy: keep +spec: + parentQueue: default-parent-queue + resources: + cpu: + quota: -1 + limit: -1 + overQuotaWeight: 1 + gpu: + quota: -1 + limit: -1 + overQuotaWeight: 1 + memory: + quota: -1 + limit: -1 + overQuotaWeight: 1 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/default-shard.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/default-shard.yaml new file mode 100644 index 000000000..da1ddad4e --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/default-shard.yaml @@ -0,0 +1,26 @@ +--- +# Source: kai-scheduler/templates/default-shard.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: kai.scheduler/v1 +kind: SchedulingShard +metadata: + name: default + annotations: + helm.sh/resource-policy: keep +spec: + partitionLabelValue: "" + placementStrategy: + gpu: binpack + cpu: binpack + plugins: + nodeplacement: + arguments: + cpu: spread + gpu: binpack + actions: + consolidation: + enabled: false + preempt: + enabled: false diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/hooks/post/post-delete-job.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/hooks/post/post-delete-job.yaml new file mode 100644 index 000000000..9d17bb268 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/hooks/post/post-delete-job.yaml @@ -0,0 +1,34 @@ +--- +# Source: kai-scheduler/templates/hooks/post/post-delete-job.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 +apiVersion: batch/v1 +kind: Job +metadata: + name: post-delete-cleanup + namespace: kai-scheduler + annotations: + "helm.sh/hook": post-delete + "helm.sh/hook-delete-policy": hook-succeeded +spec: + template: + metadata: + labels: + app.kubernetes.io/name: post-delete-cleanup + spec: + serviceAccountName: post-delete-cleanup + restartPolicy: Never + containers: + - name: deleter + image: "ghcr.io/kai-scheduler/kai-scheduler/crd-upgrader:v0.14.0" + imagePullPolicy: IfNotPresent + command: + - /bin/sh + - -c + - | + echo "Deleting deployments..." + kubectl -n kai-scheduler delete deployment --all --ignore-not-found=true + + echo "Deleting kai-config..." + kubectl delete Config kai-config --ignore-not-found + echo "Cleanup complete." diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/hooks/pre/crd-upgrader.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/hooks/pre/crd-upgrader.yaml new file mode 100644 index 000000000..2be10391d --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/hooks/pre/crd-upgrader.yaml @@ -0,0 +1,33 @@ +--- +# Source: kai-scheduler/templates/hooks/pre/crd-upgrader.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: batch/v1 +kind: Job +metadata: + name: crd-upgrader + namespace: kai-scheduler + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "2" + "helm.sh/hook-delete-policy": hook-succeeded +spec: + template: + metadata: + labels: + app: crd-upgrader + spec: + serviceAccountName: kai-scheduler-crd-manager + containers: + - name: upgrader + image: "ghcr.io/kai-scheduler/kai-scheduler/crd-upgrader:v0.14.0" + imagePullPolicy: IfNotPresent + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + runAsNonRoot: true + runAsUser: 10000 + restartPolicy: OnFailure diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/hooks/pre/topology-migration/configmap.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/hooks/pre/topology-migration/configmap.yaml new file mode 100644 index 000000000..f01f2b0a3 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/hooks/pre/topology-migration/configmap.yaml @@ -0,0 +1,73 @@ +--- +# Source: kai-scheduler/templates/hooks/pre/topology-migration/configmap.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 +apiVersion: v1 +kind: ConfigMap +metadata: + name: topology-migration-script + namespace: kai-scheduler + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "3" + "helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation +data: + migrate-topologies.sh: | + #!/bin/bash + # Copyright 2025 NVIDIA CORPORATION + # SPDX-License-Identifier: Apache-2.0 + + # Migrate Kueue Topology CRs to KAI Topology CRs + # This script is idempotent - it skips topologies that already exist in KAI + + set -e + + echo "Checking for Kueue Topology CRD..." + + # Check if Kueue Topology CRD exists + if ! kubectl get crd topologies.kueue.x-k8s.io > /dev/null 2>&1; then + echo "Kueue Topology CRD not found, skipping migration." + exit 0 + fi + + echo "Kueue Topology CRD found. Starting migration..." + + # Get all Kueue Topology names + TOPO_NAMES=$(kubectl get topologies.kueue.x-k8s.io -o jsonpath='{.items[*].metadata.name}' 2>/dev/null || echo "") + + if [ -z "$TOPO_NAMES" ]; then + echo "No Kueue Topologies found to migrate." + exit 0 + fi + + echo "Found Kueue Topologies to migrate: $TOPO_NAMES" + + # Process each topology + for TOPO_NAME in $TOPO_NAMES; do + echo "Migrating topology: $TOPO_NAME" + + # Check if KAI Topology already exists + if kubectl get topologies.kai.scheduler "$TOPO_NAME" > /dev/null 2>&1; then + echo "KAI Topology '$TOPO_NAME' already exists, skipping." + continue + fi + + # Extract the levels spec using go-template and create KAI Topology + kubectl get topologies.kueue.x-k8s.io "$TOPO_NAME" \ + -o go-template='apiVersion: kai.scheduler/v1 + kind: Topology + metadata: + name: {{.metadata.name}} + annotations: + kai.scheduler/migrated-from: "kueue.x-k8s.io" + spec: + levels: + {{- range .spec.levels}} + - nodeLabel: "{{.nodeLabel}}" + {{- end}} + ' | kubectl apply -f - + + echo "Successfully migrated topology: $TOPO_NAME" + done + + echo "Topology migration completed." diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/hooks/pre/topology-migration/job.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/hooks/pre/topology-migration/job.yaml new file mode 100644 index 000000000..64a4dcaa0 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/hooks/pre/topology-migration/job.yaml @@ -0,0 +1,41 @@ +--- +# Source: kai-scheduler/templates/hooks/pre/topology-migration/job.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 +apiVersion: batch/v1 +kind: Job +metadata: + name: topology-migration + namespace: kai-scheduler + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "4" + "helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation +spec: + template: + metadata: + labels: + app: topology-migration + spec: + serviceAccountName: kai-topology-migration + containers: + - name: migration + image: "ghcr.io/kai-scheduler/kai-scheduler/crd-upgrader:v0.14.0" + imagePullPolicy: IfNotPresent + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + runAsNonRoot: true + runAsUser: 10000 + command: ["/bin/bash", "/scripts/migrate-topologies.sh"] + volumeMounts: + - name: migration-script + mountPath: /scripts + volumes: + - name: migration-script + configMap: + name: topology-migration-script + defaultMode: 0755 + restartPolicy: OnFailure diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/hooks/pre/topology-migration/rbac.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/hooks/pre/topology-migration/rbac.yaml new file mode 100644 index 000000000..be2cf15f8 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/hooks/pre/topology-migration/rbac.yaml @@ -0,0 +1,55 @@ +--- +# Source: kai-scheduler/templates/hooks/pre/topology-migration/rbac.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: kai-topology-migration + namespace: kai-scheduler + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "1" + "helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation +--- +# Source: kai-scheduler/templates/hooks/pre/topology-migration/rbac.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: kai-topology-migration + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "1" + "helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation +rules: +# Permission to check if Kueue Topology CRD exists +- apiGroups: ["apiextensions.k8s.io"] + resources: ["customresourcedefinitions"] + resourceNames: ["topologies.kueue.x-k8s.io"] + verbs: ["get"] +# Permission to read Kueue Topologies +- apiGroups: ["kueue.x-k8s.io"] + resources: ["topologies"] + verbs: ["get", "list"] +# Permission to create/update KAI Topologies +- apiGroups: ["kai.scheduler"] + resources: ["topologies"] + verbs: ["get", "list", "create", "update", "patch"] +--- +# Source: kai-scheduler/templates/hooks/pre/topology-migration/rbac.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: kai-topology-migration + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "1" + "helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: kai-topology-migration +subjects: +- kind: ServiceAccount + name: kai-topology-migration + namespace: kai-scheduler diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/kai-config.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/kai-config.yaml new file mode 100644 index 000000000..519719e3e --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/kai-config.yaml @@ -0,0 +1,177 @@ +--- +# Source: kai-scheduler/templates/kai-config.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: kai.scheduler/v1 +kind: Config +metadata: + name: kai-config + namespace: kai-scheduler + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "3" +spec: + namespace: kai-scheduler + global: + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + runAsNonRoot: true + runAsUser: 10000 + imagesPullSecret: map[name:nvcr-pull-secret] + replicaCount: 1 + vpa: + enabled: false + resourcePolicy: + containerPolicies: + - containerName: '*' + maxAllowed: + cpu: 2 + memory: 5Gi + minAllowed: + cpu: 50m + memory: 500Mi + updatePolicy: + minReplicas: 1 + updateMode: InPlaceOrRecreate + + binder: + service: + enabled: true + image: + name: binder + repository: ghcr.io/kai-scheduler/kai-scheduler + tag: v0.14.0 + pullPolicy: IfNotPresent + resources: + limits: + cpu: 4000m + memory: 4Gi + requests: + cpu: 50m + memory: 2Gi + metricsPort: 8080 + resourceReservation: + namespace: kai-resource-reservation + serviceAccountName: kai-resource-reservation + appLabel: kai-resource-reservation + image: + name: resourcereservation + repository: ghcr.io/kai-scheduler/kai-scheduler + tag: v0.14.0 + pullPolicy: IfNotPresent + + podGrouper: + service: + enabled: true + image: + name: podgrouper + repository: ghcr.io/kai-scheduler/kai-scheduler + tag: v0.14.0 + pullPolicy: IfNotPresent + resources: + limits: + cpu: 1000m + memory: 4Gi + requests: + cpu: 50m + memory: 512Mi + + podGroupController: + service: + enabled: true + image: + name: podgroupcontroller + repository: ghcr.io/kai-scheduler/kai-scheduler + tag: v0.14.0 + pullPolicy: IfNotPresent + resources: + limits: + cpu: 1000m + memory: 1Gi + requests: + cpu: 50m + memory: 256Mi + + queueController: + service: + enabled: true + image: + name: queuecontroller + repository: ghcr.io/kai-scheduler/kai-scheduler + tag: v0.14.0 + pullPolicy: IfNotPresent + resources: + limits: + cpu: 1000m + memory: 1Gi + requests: + cpu: 50m + memory: 256Mi + + admission: + service: + enabled: true + image: + name: admission + repository: ghcr.io/kai-scheduler/kai-scheduler + tag: v0.14.0 + pullPolicy: IfNotPresent + resources: + limits: + cpu: 1000m + memory: 1Gi + requests: + cpu: 50m + memory: 256Mi + gpuSharing: false + queueLabelSelector: false + webhook: + port: 443 + targetPort: 9443 + probePort: 8081 + metricsPort: 8080 + gpuPodRuntimeClassName: "nvidia" + + nodeScaleAdjuster: + service: + enabled: false + image: + name: nodescaleadjuster + repository: ghcr.io/kai-scheduler/kai-scheduler + tag: v0.14.0 + pullPolicy: IfNotPresent + resources: + limits: + cpu: 1000m + memory: 1Gi + requests: + cpu: 50m + memory: 256Mi + args: + nodeScaleNamespace: kai-scale-adjust + scalingPodImage: + name: scalingpod + repository: ghcr.io/kai-scheduler/kai-scheduler + tag: v0.14.0 + pullPolicy: IfNotPresent + + scheduler: + service: + image: + name: scheduler + repository: ghcr.io/kai-scheduler/kai-scheduler + tag: v0.14.0 + pullPolicy: IfNotPresent + resources: + limits: + cpu: 2000m + memory: 6Gi + requests: + cpu: 500m + memory: 3Gi + schedulerService: + port: 8080 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/priorityclasses/build-preemptible.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/priorityclasses/build-preemptible.yaml new file mode 100644 index 000000000..e3a322698 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/priorityclasses/build-preemptible.yaml @@ -0,0 +1,10 @@ +--- +# Source: kai-scheduler/templates/priorityclasses/build-preemptible.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: scheduling.k8s.io/v1 +kind: PriorityClass +metadata: + name: build-preemptible +value: 75 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/priorityclasses/build.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/priorityclasses/build.yaml new file mode 100644 index 000000000..f45a22443 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/priorityclasses/build.yaml @@ -0,0 +1,10 @@ +--- +# Source: kai-scheduler/templates/priorityclasses/build.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: scheduling.k8s.io/v1 +kind: PriorityClass +metadata: + name: build +value: 100 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/priorityclasses/inference.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/priorityclasses/inference.yaml new file mode 100644 index 000000000..998c65293 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/priorityclasses/inference.yaml @@ -0,0 +1,10 @@ +--- +# Source: kai-scheduler/templates/priorityclasses/inference.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: scheduling.k8s.io/v1 +kind: PriorityClass +metadata: + name: inference +value: 125 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/priorityclasses/train.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/priorityclasses/train.yaml new file mode 100644 index 000000000..32a0454f1 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/priorityclasses/train.yaml @@ -0,0 +1,10 @@ +--- +# Source: kai-scheduler/templates/priorityclasses/train.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: scheduling.k8s.io/v1 +kind: PriorityClass +metadata: + name: train +value: 50 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/admission-binding.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/admission-binding.yaml new file mode 100644 index 000000000..89d45b6ad --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/admission-binding.yaml @@ -0,0 +1,18 @@ +--- +# Source: kai-scheduler/templates/rbac/admission-binding.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: kai-admission +subjects: + - kind: ServiceAccount + name: admission + namespace: kai-scheduler +roleRef: + kind: ClusterRole + name: kai-admission + apiGroup: rbac.authorization.k8s.io +--- +# Source: kai-scheduler/templates/rbac/admission-binding.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/admission.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/admission.yaml new file mode 100644 index 000000000..c5e4edef5 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/admission.yaml @@ -0,0 +1,47 @@ +--- +# Source: kai-scheduler/templates/rbac/admission.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: kai-admission +rules: +- apiGroups: + - "" + resources: + - configmaps + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - events + verbs: + - create + - patch + - update +- apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +--- +# Source: kai-scheduler/templates/rbac/admission.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 +# +# DO NOT EDIT - This file is auto-generated by controller-gen +# To modify RBAC permissions, edit the +kubebuilder:rbac markers in the source code +# and run 'make manifests' to regenerate this file. diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/binder-binding.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/binder-binding.yaml new file mode 100644 index 000000000..9c3322522 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/binder-binding.yaml @@ -0,0 +1,18 @@ +--- +# Source: kai-scheduler/templates/rbac/binder-binding.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: kai-binder +subjects: + - kind: ServiceAccount + name: binder + namespace: kai-scheduler +roleRef: + kind: ClusterRole + name: kai-binder + apiGroup: rbac.authorization.k8s.io +--- +# Source: kai-scheduler/templates/rbac/binder-binding.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/binder.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/binder.yaml new file mode 100644 index 000000000..72fdc9769 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/binder.yaml @@ -0,0 +1,139 @@ +--- +# Source: kai-scheduler/templates/rbac/binder.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: kai-binder +rules: +- apiGroups: + - "" + resources: + - configmaps + - pods + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - events + - pods/binding + - pods/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - "" + resources: + - nodes + - persistentvolumes + verbs: + - get + - list + - watch +- apiGroups: + - "" + resources: + - persistentvolumeclaims + verbs: + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - pods/status + verbs: + - create + - get + - list + - patch + - update + - watch +- apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - resource.k8s.io + resources: + - deviceclasses + - resourceslices + verbs: + - get + - list + - watch +- apiGroups: + - resource.k8s.io + resources: + - resourceclaims + - resourceclaims/status + verbs: + - get + - list + - patch + - update + - watch +- apiGroups: + - scheduling.run.ai + resources: + - bindrequests + verbs: + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - scheduling.run.ai + resources: + - bindrequests/finalizers + verbs: + - patch + - update +- apiGroups: + - scheduling.run.ai + resources: + - bindrequests/status + verbs: + - get + - list + - patch + - update + - watch +- apiGroups: + - storage.k8s.io + resources: + - csidrivers + - csinodes + - csistoragecapacities + - storageclasses + verbs: + - get + - list + - watch +--- +# Source: kai-scheduler/templates/rbac/binder.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 +# +# DO NOT EDIT - This file is auto-generated by controller-gen +# To modify RBAC permissions, edit the +kubebuilder:rbac markers in the source code +# and run 'make manifests' to regenerate this file. diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/crd-manager.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/crd-manager.yaml new file mode 100644 index 000000000..1d79a23a3 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/crd-manager.yaml @@ -0,0 +1,53 @@ +--- +# Source: kai-scheduler/templates/rbac/crd-manager.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: kai-scheduler-crd-manager + namespace: kai-scheduler + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "-1" + "helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation +--- +# Source: kai-scheduler/templates/rbac/crd-manager.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: kai-scheduler-crd-manager + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "-1" + "helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation +rules: +- apiGroups: ["apiextensions.k8s.io"] + resources: ["customresourcedefinitions"] + resourceNames: + - configs.kai.scheduler + - schedulingshards.kai.scheduler + - bindrequests.scheduling.run.ai + - podgroups.scheduling.run.ai + - queues.scheduling.run.ai + - topologies.kai.scheduler + verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] +--- +# Source: kai-scheduler/templates/rbac/crd-manager.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: kai-scheduler-crd-manager + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "-1" + "helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: kai-scheduler-crd-manager +subjects: +- kind: ServiceAccount + name: kai-scheduler-crd-manager + namespace: kai-scheduler diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/nodescaleadjuster-binding.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/nodescaleadjuster-binding.yaml new file mode 100644 index 000000000..594835531 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/nodescaleadjuster-binding.yaml @@ -0,0 +1,18 @@ +--- +# Source: kai-scheduler/templates/rbac/nodescaleadjuster-binding.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: kai-node-scale-adjuster +subjects: + - kind: ServiceAccount + name: node-scale-adjuster + namespace: kai-scheduler +roleRef: + kind: ClusterRole + name: kai-node-scale-adjuster + apiGroup: rbac.authorization.k8s.io +--- +# Source: kai-scheduler/templates/rbac/nodescaleadjuster-binding.yaml +# Copyright 2025 NVIDIA CORPORATION + # SPDX-License-Identifier: Apache-2.0 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/nodescaleadjuster.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/nodescaleadjuster.yaml new file mode 100644 index 000000000..2eb9f93e9 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/nodescaleadjuster.yaml @@ -0,0 +1,57 @@ +--- +# Source: kai-scheduler/templates/rbac/nodescaleadjuster.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: kai-node-scale-adjuster +rules: +- apiGroups: + - "" + resources: + - configmaps + - pods + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - events + - pods/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - "" + resources: + - pods/status + verbs: + - get + - patch + - update +- apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +--- +# Source: kai-scheduler/templates/rbac/nodescaleadjuster.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 +# +# DO NOT EDIT - This file is auto-generated by controller-gen +# To modify RBAC permissions, edit the +kubebuilder:rbac markers in the source code +# and run 'make manifests' to regenerate this file. diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/operator-binding.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/operator-binding.yaml new file mode 100644 index 000000000..bb0ff84dc --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/operator-binding.yaml @@ -0,0 +1,18 @@ +--- +# Source: kai-scheduler/templates/rbac/operator-binding.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: kai-operator +subjects: + - kind: ServiceAccount + name: kai-operator + namespace: kai-scheduler +roleRef: + kind: ClusterRole + name: kai-operator + apiGroup: rbac.authorization.k8s.io +--- +# Source: kai-scheduler/templates/rbac/operator-binding.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/operator.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/operator.yaml new file mode 100644 index 000000000..2f4c903c6 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/operator.yaml @@ -0,0 +1,179 @@ +--- +# Source: kai-scheduler/templates/rbac/operator.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: kai-operator +rules: +- apiGroups: + - "" + resources: + - configmaps + - endpoints + - persistentvolumeclaims + - pods + - secrets + - serviceaccounts + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - nodes + verbs: + - get + - list + - watch +- apiGroups: + - admissionregistration.k8s.io + resources: + - mutatingwebhookconfigurations + - validatingwebhookconfigurations + verbs: + - create + - get + - list + - watch +- apiGroups: + - admissionregistration.k8s.io + resourceNames: + - kai-podgroup-validation-v2alpha2 + - kai-queue-validation-v2 + - mutating-kai-admission + - validating-kai-admission + resources: + - mutatingwebhookconfigurations + - validatingwebhookconfigurations + verbs: + - delete + - patch + - update +- apiGroups: + - apiextensions.k8s.io + resources: + - customresourcedefinitions + verbs: + - create + - get + - list + - watch +- apiGroups: + - apiextensions.k8s.io + resourceNames: + - queues.scheduling.run.ai + resources: + - customresourcedefinitions + verbs: + - delete + - patch + - update +- apiGroups: + - apps + resources: + - daemonsets + - deployments + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - autoscaling.k8s.io + resources: + - verticalpodautoscalers + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - kai.scheduler + resources: + - configs + - schedulingshards + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - kai.scheduler + resources: + - configs/finalizers + - schedulingshards/finalizers + verbs: + - update +- apiGroups: + - kai.scheduler + resources: + - configs/status + - schedulingshards/status + verbs: + - get + - patch + - update +- apiGroups: + - monitoring.coreos.com + resources: + - prometheuses + - servicemonitors + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - nvidia.com + resources: + - clusterpolicies + verbs: + - get + - list + - watch +- apiGroups: + - scheduling.run.ai + resources: + - queues + verbs: + - get + - list + - watch +--- +# Source: kai-scheduler/templates/rbac/operator.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 +# +# DO NOT EDIT - This file is auto-generated by controller-gen +# To modify RBAC permissions, edit the +kubebuilder:rbac markers in the source code +# and run 'make manifests' to regenerate this file. diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/podgroupcontroller-binding.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/podgroupcontroller-binding.yaml new file mode 100644 index 000000000..978c8db4f --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/podgroupcontroller-binding.yaml @@ -0,0 +1,18 @@ +--- +# Source: kai-scheduler/templates/rbac/podgroupcontroller-binding.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: kai-podgroup-controller +subjects: + - kind: ServiceAccount + name: podgroup-controller + namespace: kai-scheduler +roleRef: + kind: ClusterRole + name: kai-podgroup-controller + apiGroup: rbac.authorization.k8s.io +--- +# Source: kai-scheduler/templates/rbac/podgroupcontroller-binding.yaml +# Copyright 2025 NVIDIA CORPORATION + # SPDX-License-Identifier: Apache-2.0 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/podgroupcontroller.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/podgroupcontroller.yaml new file mode 100644 index 000000000..2278c724b --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/podgroupcontroller.yaml @@ -0,0 +1,83 @@ +--- +# Source: kai-scheduler/templates/rbac/podgroupcontroller.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: kai-podgroup-controller +rules: +- apiGroups: + - "" + resources: + - configmaps + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - nodes + - pods + - pods/status + verbs: + - get + - list + - watch +- apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - resource.k8s.io + resources: + - resourceclaims + verbs: + - get + - list + - watch +- apiGroups: + - scheduling.k8s.io + resources: + - priorityclasses + verbs: + - get + - list + - watch +- apiGroups: + - scheduling.run.ai + resources: + - podgroups + verbs: + - get + - list + - watch +- apiGroups: + - scheduling.run.ai + resources: + - podgroups/status + verbs: + - get + - list + - patch + - update + - watch +--- +# Source: kai-scheduler/templates/rbac/podgroupcontroller.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 +# +# DO NOT EDIT - This file is auto-generated by controller-gen +# To modify RBAC permissions, edit the +kubebuilder:rbac markers in the source code +# and run 'make manifests' to regenerate this file. diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/podgrouper-binding.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/podgrouper-binding.yaml new file mode 100644 index 000000000..42d5c12f7 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/podgrouper-binding.yaml @@ -0,0 +1,18 @@ +--- +# Source: kai-scheduler/templates/rbac/podgrouper-binding.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: kai-podgrouper +subjects: + - kind: ServiceAccount + name: pod-grouper + namespace: kai-scheduler +roleRef: + kind: ClusterRole + name: kai-podgrouper + apiGroup: rbac.authorization.k8s.io +--- +# Source: kai-scheduler/templates/rbac/podgrouper-binding.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/podgrouper.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/podgrouper.yaml new file mode 100644 index 000000000..804f4ff34 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/podgrouper.yaml @@ -0,0 +1,415 @@ +--- +# Source: kai-scheduler/templates/rbac/podgrouper.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: kai-podgrouper +rules: +- apiGroups: + - "" + resources: + - configmaps + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - events + - pods + verbs: + - create + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - namespaces + verbs: + - get + - list + - watch +- apiGroups: + - "" + resources: + - pods/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - amlarc.azureml.com + resources: + - amljobs + verbs: + - get + - list + - watch +- apiGroups: + - amlarc.azureml.com + resources: + - amljobs/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - apps + resources: + - deployments + - replicasets + - statefulsets + verbs: + - get + - list + - watch +- apiGroups: + - apps + resources: + - deployments/finalizers + - replicasets/finalizers + - statefulsets/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - argoproj.io + resources: + - workflows + verbs: + - get + - list + - watch +- apiGroups: + - argoproj.io + resources: + - workflows/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - batch + resources: + - cronjobs + - jobs + verbs: + - get + - list + - watch +- apiGroups: + - batch + resources: + - cronjobs/finalizers + - jobs/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - egx.nvidia.io + resources: + - spotrequests + verbs: + - get + - list + - watch +- apiGroups: + - egx.nvidia.io + resources: + - spotrequests/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - grove.io + resources: + - podcliques + - podcliquescalinggroups + - podcliquesets + - podgangsets + verbs: + - get + - list + - watch +- apiGroups: + - grove.io + resources: + - podcliques/finalizers + - podcliquescalinggroups/finalizers + - podcliquesets/finalizers + - podgangsets/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - jobset.x-k8s.io + resources: + - jobsets + verbs: + - get + - list + - watch +- apiGroups: + - jobset.x-k8s.io + resources: + - jobsets/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - kubeflow.org + resources: + - jaxjobs + - mpijobs + - notebooks + - pytorchjobs + - scheduledworkflows + - tfjobs + - xgboostjobs + verbs: + - get + - list + - watch +- apiGroups: + - kubeflow.org + resources: + - jaxjobs/finalizers + - mpijobs/finalizers + - notebooks/finalizers + - pytorchjobs/finalizers + - scheduledworkflows/finalizers + - tfjobs/finalizers + - xgboostjobs/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - kubevirt.io + resources: + - virtualmachineinstances + - virtualmachines + verbs: + - get + - list + - watch +- apiGroups: + - kubevirt.io + resources: + - virtualmachineinstances/finalizers + - virtualmachines/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - leaderworkerset.x-k8s.io + resources: + - leaderworkersets + verbs: + - get + - list + - watch +- apiGroups: + - leaderworkerset.x-k8s.io + resources: + - leaderworkersets/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - machinelearning.seldon.io + resources: + - seldondeployments + verbs: + - get + - list + - watch +- apiGroups: + - machinelearning.seldon.io + resources: + - seldondeployments/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - ray.io + resources: + - rayclusters + - rayjobs + - rayservices + verbs: + - get + - list + - watch +- apiGroups: + - ray.io + resources: + - rayclusters/finalizers + - rayjobs/finalizers + - rayservices/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - run.ai + resources: + - distributedinferenceworkloads + - distributedworkloads + - inferenceworkloads + - interactiveworkloads + - runaijobs + - trainingworkloads + verbs: + - get + - list + - watch +- apiGroups: + - run.ai + resources: + - runaijobs/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - scheduler.grove.io + resources: + - podgangs + verbs: + - get + - list + - watch +- apiGroups: + - scheduler.grove.io + resources: + - podgangs/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - scheduling.k8s.io + resources: + - priorityclasses + verbs: + - get + - list + - watch +- apiGroups: + - scheduling.run.ai + resources: + - podgroups + verbs: + - create + - get + - list + - patch + - update + - watch +- apiGroups: + - serving.knative.dev + resources: + - configurations + - revisions + - services + verbs: + - get + - list + - watch +- apiGroups: + - serving.knative.dev + resources: + - configurations/finalizers + - revisions/finalizers + - services/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - tekton.dev + resources: + - pipelineruns + - taskruns + verbs: + - get + - list + - watch +- apiGroups: + - tekton.dev + resources: + - pipelineruns/finalizers + - taskruns/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - trainer.kubeflow.org + resources: + - trainjobs + verbs: + - get + - list + - watch +- apiGroups: + - trainer.kubeflow.org + resources: + - trainjobs/finalizers + verbs: + - create + - patch + - update +- apiGroups: + - workspace.devfile.io + resources: + - devworkspaces + verbs: + - get + - list + - watch +- apiGroups: + - workspace.devfile.io + resources: + - devworkspaces/finalizers + verbs: + - create + - patch + - update +--- +# Source: kai-scheduler/templates/rbac/podgrouper.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 +# +# DO NOT EDIT - This file is auto-generated by controller-gen +# To modify RBAC permissions, edit the +kubebuilder:rbac markers in the source code +# and run 'make manifests' to regenerate this file. diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/post-delete-binding.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/post-delete-binding.yaml new file mode 100644 index 000000000..664a53505 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/post-delete-binding.yaml @@ -0,0 +1,21 @@ +--- +# Source: kai-scheduler/templates/rbac/post-delete-binding.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 +--- +# Source: kai-scheduler/templates/rbac/post-delete-binding.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: post-delete-cleanup + annotations: + "helm.sh/hook": post-delete + "helm.sh/hook-delete-policy": hook-succeeded +subjects: + - kind: ServiceAccount + name: post-delete-cleanup + namespace: kai-scheduler +roleRef: + kind: ClusterRole + name: post-delete-cleanup + apiGroup: rbac.authorization.k8s.io diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/post-delete-clusterrole.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/post-delete-clusterrole.yaml new file mode 100644 index 000000000..c319d1b51 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/post-delete-clusterrole.yaml @@ -0,0 +1,29 @@ +--- +# Source: kai-scheduler/templates/rbac/post-delete-clusterrole.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 +--- +# Source: kai-scheduler/templates/rbac/post-delete-clusterrole.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: post-delete-cleanup + annotations: + "helm.sh/hook": post-delete + "helm.sh/hook-delete-policy": hook-succeeded +rules: +- apiGroups: + - apps + resources: + - deployments + verbs: + - delete + - get + - list + - watch +- apiGroups: + - kai.scheduler + resources: + - configs + verbs: + - delete diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/prometheus-binding.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/prometheus-binding.yaml new file mode 100644 index 000000000..c2deeb167 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/prometheus-binding.yaml @@ -0,0 +1,18 @@ +--- +# Source: kai-scheduler/templates/rbac/prometheus-binding.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: kai-prometheus +subjects: + - kind: ServiceAccount + name: prometheus + namespace: kai-scheduler +roleRef: + kind: ClusterRole + name: kai-prometheus + apiGroup: rbac.authorization.k8s.io +--- +# Source: kai-scheduler/templates/rbac/prometheus-binding.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/prometheus-pod.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/prometheus-pod.yaml new file mode 100644 index 000000000..6ca0e6515 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/prometheus-pod.yaml @@ -0,0 +1,44 @@ +--- +# Source: kai-scheduler/templates/rbac/prometheus-pod.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: kai-prometheus +rules: +- apiGroups: + - "" + resources: + - nodes + - nodes/metrics + - services + - endpoints + - pods + verbs: + - get + - list + - watch +- apiGroups: + - discovery.k8s.io + resources: + - endpointslices + verbs: + - get + - list + - watch +- apiGroups: + - networking.k8s.io + resources: + - ingresses + verbs: + - get + - list + - watch +- nonResourceURLs: + - /metrics + - /metrics/cadvisor + verbs: + - get +--- +# Source: kai-scheduler/templates/rbac/prometheus-pod.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/queuecontroller-binding.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/queuecontroller-binding.yaml new file mode 100644 index 000000000..f270e5d69 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/queuecontroller-binding.yaml @@ -0,0 +1,17 @@ +--- +# Source: kai-scheduler/templates/rbac/queuecontroller-binding.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: kai-queuecontroller +subjects: +- kind: ServiceAccount + name: queue-controller + namespace: kai-scheduler +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: queuecontroller diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/queuecontroller.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/queuecontroller.yaml new file mode 100644 index 000000000..1a7e5dac0 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/queuecontroller.yaml @@ -0,0 +1,71 @@ +--- +# Source: kai-scheduler/templates/rbac/queuecontroller.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: queuecontroller +rules: +- apiGroups: + - "" + resources: + - configmaps + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - scheduling.run.ai + resources: + - podgroups + verbs: + - get + - list + - watch +- apiGroups: + - scheduling.run.ai + resources: + - queues + verbs: + - get + - list + - patch + - update + - watch +- apiGroups: + - scheduling.run.ai + resources: + - queues/finalizers + verbs: + - update +- apiGroups: + - scheduling.run.ai + resources: + - queues/status + verbs: + - get + - patch + - update +--- +# Source: kai-scheduler/templates/rbac/queuecontroller.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 +# +# DO NOT EDIT - This file is auto-generated by controller-gen +# To modify RBAC permissions, edit the +kubebuilder:rbac markers in the source code +# and run 'make manifests' to regenerate this file. diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/resourcereservation-binding.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/resourcereservation-binding.yaml new file mode 100644 index 000000000..afd02a33d --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/resourcereservation-binding.yaml @@ -0,0 +1,19 @@ +--- +# Source: kai-scheduler/templates/rbac/resourcereservation-binding.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: kai-resource-reservation + namespace: kai-resource-reservation +subjects: + - kind: ServiceAccount + name: kai-resource-reservation + namespace: kai-resource-reservation +roleRef: + kind: ClusterRole + name: kai-resource-reservation + apiGroup: rbac.authorization.k8s.io +--- +# Source: kai-scheduler/templates/rbac/resourcereservation-binding.yaml +# Copyright 2025 NVIDIA CORPORATION + # SPDX-License-Identifier: Apache-2.0 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/resourcereservation.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/resourcereservation.yaml new file mode 100644 index 000000000..d3eb9c7bd --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/resourcereservation.yaml @@ -0,0 +1,25 @@ +--- +# Source: kai-scheduler/templates/rbac/resourcereservation.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: kai-resource-reservation +rules: +- apiGroups: + - "" + resources: + - pods + verbs: + - get + - list + - patch + - update + - watch +--- +# Source: kai-scheduler/templates/rbac/resourcereservation.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 +# +# DO NOT EDIT - This file is auto-generated by controller-gen +# To modify RBAC permissions, edit the +kubebuilder:rbac markers in the source code +# and run 'make manifests' to regenerate this file. diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/scc.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/scc.yaml new file mode 100644 index 000000000..c5368e037 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/scc.yaml @@ -0,0 +1,4 @@ +--- +# Source: kai-scheduler/templates/rbac/scc.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/scheduler-binding.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/scheduler-binding.yaml new file mode 100644 index 000000000..59555c044 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/scheduler-binding.yaml @@ -0,0 +1,18 @@ +--- +# Source: kai-scheduler/templates/rbac/scheduler-binding.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: kai-scheduler +subjects: + - kind: ServiceAccount + name: scheduler + namespace: kai-scheduler +roleRef: + kind: ClusterRole + name: kai-scheduler + apiGroup: rbac.authorization.k8s.io +--- +# Source: kai-scheduler/templates/rbac/scheduler-binding.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/scheduler.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/scheduler.yaml new file mode 100644 index 000000000..8ea5169c7 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/rbac/scheduler.yaml @@ -0,0 +1,142 @@ +--- +# Source: kai-scheduler/templates/rbac/scheduler.yaml +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: kai-scheduler +rules: +- apiGroups: + - "" + resources: + - configmaps + - events + - pods/status + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - namespaces + - nodes + - persistentvolumeclaims + - persistentvolumes + verbs: + - get + - list + - watch +- apiGroups: + - "" + resources: + - pods + verbs: + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - "" + resources: + - pods/finalizers + verbs: + - create + - delete + - get + - list + - patch + - update +- apiGroups: + - coordination.k8s.io + resources: + - leases + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - kai.scheduler + resources: + - topologies + verbs: + - get + - list + - watch +- apiGroups: + - resource.k8s.io + resources: + - deviceclasses + - resourceclaims + - resourceslices + verbs: + - get + - list + - watch +- apiGroups: + - scheduling.k8s.io + resources: + - priorityclasses + verbs: + - get + - list + - watch +- apiGroups: + - scheduling.run.ai + resources: + - bindrequests + - podgroups/status + verbs: + - create + - delete + - get + - list + - patch + - update + - watch +- apiGroups: + - scheduling.run.ai + resources: + - podgroups + verbs: + - get + - list + - patch + - update + - watch +- apiGroups: + - scheduling.run.ai + resources: + - queues + verbs: + - get + - list + - watch +- apiGroups: + - storage.k8s.io + resources: + - csidrivers + - csinodes + - csistoragecapacities + - storageclasses + verbs: + - get + - list + - watch +--- +# Source: kai-scheduler/templates/rbac/scheduler.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 +# +# DO NOT EDIT - This file is auto-generated by controller-gen +# To modify RBAC permissions, edit the +kubebuilder:rbac markers in the source code +# and run 'make manifests' to regenerate this file. diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/services/operator-serviceaccount.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/services/operator-serviceaccount.yaml new file mode 100644 index 000000000..a61c5c187 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/services/operator-serviceaccount.yaml @@ -0,0 +1,12 @@ +--- +# Source: kai-scheduler/templates/services/operator-serviceaccount.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: v1 +kind: ServiceAccount +metadata: + name: kai-operator + namespace: kai-scheduler + labels: + app: kai-operator diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/services/operator.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/services/operator.yaml new file mode 100644 index 000000000..a6d155579 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/services/operator.yaml @@ -0,0 +1,73 @@ +--- +# Source: kai-scheduler/templates/services/operator.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: apps/v1 +kind: Deployment +metadata: + name: kai-operator + namespace: kai-scheduler + labels: + app: kai-operator +spec: + replicas: 1 + selector: + matchLabels: + app: kai-operator + template: + metadata: + labels: + app: kai-operator + spec: + serviceAccountName: kai-operator + containers: + - name: operator + image: ghcr.io/kai-scheduler/kai-scheduler/operator:v0.14.0 + imagePullPolicy: IfNotPresent + resources: + limits: + cpu: 1000m + memory: 1Gi + requests: + cpu: 50m + memory: 256Mi + args: + - --metrics-bind-address=:8080 + - --health-probe-bind-address=:8081 + - --leader-elect=false + - --qps=50 + - --burst=300 + - --namespace=kai-scheduler + env: + - name: WATCH_NAMESPACE + value: kai-scheduler + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: OPERATOR_NAME + value: "kai-operator" + - name: MS_REPOSITORY + value: ghcr.io/kai-scheduler/kai-scheduler + - name: MS_TAG + value: v0.14.0 + ports: + - containerPort: 8080 + name: metrics + - containerPort: 8081 + name: probe + livenessProbe: + httpGet: + path: /healthz + port: 8081 + initialDelaySeconds: 15 + periodSeconds: 20 + readinessProbe: + httpGet: + path: /readyz + port: 8081 + initialDelaySeconds: 5 + periodSeconds: 10 + imagePullSecrets: + - name: nvcr-pull-secret diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/services/post-delete-serviceaccount.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/services/post-delete-serviceaccount.yaml new file mode 100644 index 000000000..26c1cb238 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/services/post-delete-serviceaccount.yaml @@ -0,0 +1,14 @@ +--- +# Source: kai-scheduler/templates/services/post-delete-serviceaccount.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 +apiVersion: v1 +kind: ServiceAccount +metadata: + name: post-delete-cleanup + namespace: kai-scheduler + labels: + app: kai-operator + annotations: + "helm.sh/hook": post-delete + "helm.sh/hook-delete-policy": hook-succeeded diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/services/resourcereservation-namespace.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/services/resourcereservation-namespace.yaml new file mode 100644 index 000000000..8f745726b --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/services/resourcereservation-namespace.yaml @@ -0,0 +1,10 @@ +--- +# Source: kai-scheduler/templates/services/resourcereservation-namespace.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 +apiVersion: v1 +kind: Namespace +metadata: + name: kai-resource-reservation + annotations: + helm.sh/resource-policy: keep diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/services/resourcereservation-serviceaccount.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/services/resourcereservation-serviceaccount.yaml new file mode 100644 index 000000000..de2e7926b --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/services/resourcereservation-serviceaccount.yaml @@ -0,0 +1,12 @@ +--- +# Source: kai-scheduler/templates/services/resourcereservation-serviceaccount.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 + +apiVersion: v1 +kind: ServiceAccount +metadata: + name: kai-resource-reservation + namespace: kai-resource-reservation +imagePullSecrets: + - name: nvcr-pull-secret diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/services/scalingpod-namespace.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/services/scalingpod-namespace.yaml new file mode 100644 index 000000000..e79c88bc6 --- /dev/null +++ b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/01-dependencies.yaml-a814432c-kai-scheduler/kai-scheduler/templates/services/scalingpod-namespace.yaml @@ -0,0 +1,4 @@ +--- +# Source: kai-scheduler/templates/services/scalingpod-namespace.yaml +# Copyright 2025 NVIDIA CORPORATION +# SPDX-License-Identifier: Apache-2.0 diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/agent-config-merge-cm.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/agent-config-merge-cm.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/agent-config-merge-cm.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/agent-config-merge-cm.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/chart-defaults-nvcfbackend-cm.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/chart-defaults-nvcfbackend-cm.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/chart-defaults-nvcfbackend-cm.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/chart-defaults-nvcfbackend-cm.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/cluster-validator-network-checks-cm.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/cluster-validator-network-checks-cm.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/cluster-validator-network-checks-cm.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/cluster-validator-network-checks-cm.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/crds/nvidia.io_nvcfbackends_crd.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/crds/nvidia.io_nvcfbackends_crd.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/crds/nvidia.io_nvcfbackends_crd.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/crds/nvidia.io_nvcfbackends_crd.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/cronjob.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/cronjob.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/cronjob.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/cronjob.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/custom-annotations-configmap.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/custom-annotations-configmap.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/custom-annotations-configmap.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/custom-annotations-configmap.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/custom-network-policies-configmap.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/custom-network-policies-configmap.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/custom-network-policies-configmap.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/custom-network-policies-configmap.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/deployment.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/deployment.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/deployment.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/deployment.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/gpu-profiling-config-configmap.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/gpu-profiling-config-configmap.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/gpu-profiling-config-configmap.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/gpu-profiling-config-configmap.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/helm-managed-nvcfbackend-cm.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/helm-managed-nvcfbackend-cm.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/helm-managed-nvcfbackend-cm.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/helm-managed-nvcfbackend-cm.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/ngc-service-key.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/ngc-service-key.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/ngc-service-key.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/ngc-service-key.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/nvca-operator_rq.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/nvca-operator_rq.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/nvca-operator_rq.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/nvca-operator_rq.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/pre-delete-cleanup-job.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/pre-delete-cleanup-job.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/pre-delete-cleanup-job.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/pre-delete-cleanup-job.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/pre-delete-cleanup-rbac.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/pre-delete-cleanup-rbac.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/pre-delete-cleanup-rbac.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/pre-delete-cleanup-rbac.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/rbac.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/rbac.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/rbac.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/rbac.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/rbac_allowed_extra_types.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/rbac_allowed_extra_types.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/rbac_allowed_extra_types.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/rbac_allowed_extra_types.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/role.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/role.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/role.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/role.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/role_binding.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/role_binding.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/role_binding.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/role_binding.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/sa.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/sa.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/sa.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/sa.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/self-managed-nvcfbackend-cm.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/self-managed-nvcfbackend-cm.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/self-managed-nvcfbackend-cm.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/self-managed-nvcfbackend-cm.yaml diff --git a/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/shutdown-sentinel.yaml b/deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/shutdown-sentinel.yaml similarity index 100% rename from deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-31ea6bf9-nvca-operator/helm-nvca-operator/templates/shutdown-sentinel.yaml rename to deploy/stacks/nvcf-compute-plane/testdata/golden/local/02-nvca.yaml-8e61e521-nvca-operator/helm-nvca-operator/templates/shutdown-sentinel.yaml diff --git a/docs/user/cluster-management/kai-scheduler.md b/docs/user/cluster-management/kai-scheduler.md index 809b8fd6b..78af4db19 100644 --- a/docs/user/cluster-management/kai-scheduler.md +++ b/docs/user/cluster-management/kai-scheduler.md @@ -7,10 +7,14 @@ cluster. NVCF Workloads deployed will be automatically BinPacked upon this clust **KAI Scheduler Installation** -Upgrade to latest [KAI Scheduler release](https://github.com/kai-scheduler/KAI-Scheduler/releases) is recommended to get latest fixes and security patches + Upgrade to latest [KAI Scheduler release](https://github.com/kai-scheduler/KAI-Scheduler/releases) is recommended to get latest fixes and security patches +When you enable `addons.kaiScheduler.enabled` in the `nvcf-compute-plane` Helmfile stack, the stack installs KAI Scheduler for you (release and namespace `kai-scheduler`). Enable that flag whenever you enable Grove or Dynamo. Skip the manual install below in that case. + +Use the manual install when you need KAI without the compute-plane add-on, for example when enabling only the NVCA `KAIScheduler` feature gate. + NVCA's KAI scheduler integration expects default queues to exist with names `default-parent-queue` (parent) and `default-queue` (child); other queues may exist in the cluster.